Commit graph

521 commits

Author SHA1 Message Date
Markus Goetz
a0521caf52 CSync: Allow files with 0 mtime #1603
I don't see a reason we should not allow this. We detect
a "failed stat reply" in other ways. Maybe this was a legacy
csync decision somewhow that is not valid anymore?
2016-04-14 18:15:16 +02:00
Christian Kamm
6f454feb39 Fix hidden file handling #4655
There were two issues:

* With the refactoring of how Folder and SyncEngine relate, the
  ignore_hidden_files flag on the CSync context was reset after
  each sync run and not updated from the configuration again.

* The folder watcher failed to enumerate hidden folders and thus
  didn't watch for changes inside them. (linux only)
2016-04-12 11:59:07 +02:00
Christian Kamm
ceb4a23dab Fix compile with strict C flags 2016-03-18 12:41:57 +01:00
Christian Kamm
d7bd1300a8 Ignores: expand escapes #4568 2016-03-17 14:31:53 +01:00
Jocelyn Turcotte
7561f5c717 Add comment and cleanup the csync exclude list code
Removed the csync_add_exclude_list function that isn't use anywhere
anymore.
2016-03-03 20:26:48 +01:00
Jocelyn Turcotte
b8227afcaa Move the csync_context creation in SyncEngine
The creation doesn't need to be separated from the SyncEngine anymore.
This allows the SyncEngine to be created in fewer steps if we want to
use it in tests.

This moves most of the direct csync code from Folder into the SyncEngine.
The exclude file logic for the context has been wrapped using the
existing ExcludedFiles class as well.
2016-02-25 20:53:13 +01:00
Jocelyn Turcotte
b8dee63d7a Remove superfluous error checking from csync_create and csync_init
Given that we control all call sites, the only way that this can fail is during
OOM. Also remove the code in csync itself to make sure that it's obvious that
any new error case wouldn't be handled by call sites.
2016-02-25 20:53:13 +01:00
Hefee
28b694b170 Merge branch '2.1' 2016-02-20 10:34:34 +01:00
Hefee
74f74e0363 fix typo occured->occurred 2016-02-20 10:33:13 +01:00
Olivier Goffart
93308faeb9 Merge remote-tracking branch 'origin/2.1'
Conflicts:
	doc/building.rst
2016-02-08 14:28:25 +01:00
Olivier Goffart
6b643c7501 Discovery: don't ignore recall file #4420
The ".sys.admin#recall#" is the recall file and should not be ignored
even if hidden.

The remote discovery do not need to detect hidden files because it
is already detected by csync in all cases. So this avoid code duplication
2016-02-08 12:07:34 +01:00
Hefee
aa8b772bff Don't install tests and cmocka lib.
Neither tests nor the libcmocka needs to be installed globally.
2016-02-05 14:14:07 +01:00
Klaas Freitag
57c7727479 csync_update: Remove unneeded checks of previous commit. 2016-01-29 10:43:31 +01:00
Klaas Freitag
28907ec0c3 csync_update: Handle comparision of file types properly.
Note that the structs use different enums for the file types, unfortunately.
2016-01-27 14:11:37 +01:00
Klaas Freitag
7ddfa79950 csync_update: Use the csync defines rather than plain numbers. 2016-01-27 14:10:52 +01:00
Daniel Molkentin
dd76d72d61 Merge remote-tracking branch 'origin/2.1' 2016-01-26 14:09:38 +01:00
Christian Kamm
62ded39416 Update: Check checksum for renamed files #2983
This currently doesn't do much because we only compute content
checksums for .eml files.
2016-01-20 13:21:48 +01:00
Klaas Freitag
a41fbc0454 csync_private: Fix comment about size of file id buffer 2016-01-15 12:14:46 +01:00
Klaas Freitag
d89edc35d2 csync-tests: Added some directory wildcard tests.
This was to verify client issue #1558
2016-01-10 12:04:07 +01:00
Christian Kamm
abf5a5ad1e Propagation: Fix dir <-> file changes propagating to server #4302
* Ensure every time a file becomes a directory or the other way around
  the item is flagged as INSTRUCTION_TYPE_CHANGE.
* Delete the badly-typed entity if necessary in the propagation jobs.
2016-01-08 05:38:08 +01:00
Christian Kamm
5cc4c03b6a Propagator: Deal with files becoming directories #4302
This needed adjustments in reconcile, to mark the item as SYNC
as well as additions to the LocalMkdir job.
2016-01-05 10:26:41 +01:00
Olivier Goffart
cf5b1e401c csync_vio_local_stat: Win: fetch mtime and size if not previously fetched
Since owncloud 2.1, csync_vio_local_stat was optimized because readdir
would already fetch most of the info.  This works for the discovery,
but not later.  And we used this function later for symliks and co.

So this fixes the .lnk on windows

Issue #4300
2015-12-21 14:25:30 +01:00
Olivier Goffart
685f2a259f csync: remove PLUGIN install patch from cmake files
This conflicts with the KDE_INSTALL_PLUGINDIR and prevent the
dolphin plugin to be installed at the right location.

Since csync no longer have plugin, this can be removed
2015-12-02 13:13:41 +01:00
Christian Kamm
1aa570326e Tests: Add a perl test for eml checksums #3235 2015-11-25 12:48:24 +01:00
Christian Kamm
2458f07ca1 Checksums: Reuse the discovery checksum where possible 2015-11-23 14:13:10 +01:00
Christian Kamm
3b7927366a csync_update: Remove disabled inode check 2015-11-23 12:01:42 +01:00
Christian Kamm
a25f094c4c Checksums: Don't reupload if size and checksum are unchanged #3235
* Compute the content checksum (in addition to the optional
  transmission checksum) during upload (.eml files only)

* Add hook to compute and compare the checksum in csync_update

* Add content checksum to database, remove transmission checksum
2015-11-23 11:59:56 +01:00
Klaas Freitag
be88e5a2c3 csync_update: refactor a bit and remove redundant code. 2015-11-18 15:31:55 +01:00
Klaas Freitag
e7ad7d405c csync_update: Add missing comment about 403 forbidden handling. 2015-11-18 15:31:55 +01:00
Klaas Freitag
62d26814b2 SyncEngine: Handle 403 Forbidden properly.
A 403 is a reply code sent from the file firewall to indicate that
this directory is forbidden to use for the user.

The patch handles it by setting the state to IGNORED.

This addresses #3490
2015-11-18 15:31:55 +01:00
Christian Kamm
beb970646b Fix csync_exclude test
The exclude file was reorganized in
b6aa18bfbc
2015-11-11 09:42:31 +01:00
Christian Kamm
d9f8edd259 Hidden files: Consider .* hidden everywhere #4023
This seems to be the only place where we did this only on
non-windows OSes.
2015-10-29 11:59:46 +01:00
Markus Goetz
a6c9e8c5b4 Merge branch 'kill_legacy_propagator' 2015-10-28 11:06:50 +01:00
Christian Kamm
c81b02c7d9 csync tests: Remove unused function 2015-10-28 10:04:15 +01:00
Christian Kamm
d2e5ba123d Fix compile after pull request merge
It conflicted with the optimization in
f18b40f7e7
2015-10-28 09:53:06 +01:00
Klaas Freitag
0c9568f6dc Merge pull request #4004 from owncloud/fix_hidden_detection
Fix hidden detection
2015-10-28 09:28:35 +01:00
Klaas Freitag
6b71273380 Discovery: Fix detection of hidden files.
In the discovery phase we want to detect that dot-files are hidden
on Linux and Mac.

This fixes strange behaviour seen in issue #3980
2015-10-26 14:24:05 +01:00
Olivier Goffart
f18b40f7e7 csync_vio: reuse the information from readdir in stat
On unix we don't safe much (otherwise csync would have been
designed differently).
On windows however, the readdir already fetch all the info, so we
can as well use it.

We still have to query for the file id but we might optimize that later
2015-10-22 18:46:41 +02:00
Olivier Goffart
91525a7d33 csync_exclude: Don't ignore invalid char client side (#3736)
If the server does not support it, then the server will reply with an error
2015-10-22 12:32:53 +02:00
Olivier Goffart
597d36dcf2 csync_statedb: Use the index in csync_statedb_get_below_path
Make an index from the path, and make a query that uses the index
2015-10-21 16:38:26 +02:00
Olivier Goffart
9c388787bb csync_update: Don't fetch the etag in the local discovery from the DB
We don't need it, and it's slow.
This saves a lot of DB queries

(Also replaced a strlen>0 with a faster check)
2015-10-21 16:38:26 +02:00
Olivier Goffart
71827549d6 csync_exclude: Use PathMatchSpecA instead of PathMatchSpecW
So we avoid lots of memory allocation.
We can work with char* directly since both the pattern and the file
name are in UTF-8 and there is no need to understand unicode for
such pattern.

(In fact, '?' would not match anyore non-ascii characters, but I
don't think that's a problem. I don't think anyone use '?' in its
exclude list. And the two allocations per call to csync_fnmatch are
really worth getting rid of)
2015-10-20 18:38:47 +02:00
Olivier Goffart
de5de7acc5 csync_exclude: Optimize
Avoid alocations as much as possible
2015-10-20 18:38:47 +02:00
Markus Goetz
c8590c4468 Remove legacy propagator and neon
The code was already uneeded/unbuilt on Windows and OS X.
2015-10-20 17:57:43 +02:00
Olivier Goffart
b7c927f349 csync: Fix warning the bit field need to be unsigned 2015-10-19 10:13:11 +02:00
Olivier Goffart
83edb7fb8a csync: fix compilation error with -pedentic 2015-10-16 11:45:36 +02:00
Olivier Goffart
4df828d0b0 csync_file_stat_s: Save a bit of memory 2015-10-16 09:56:26 +02:00
Hugo Gonzalez Labrador
4d1d6dff3b Increased fileID size to 36 bytes
This is the size used by UUIDs
2015-10-16 09:27:19 +02:00
Klaas Freitag
e7e0e5b57f win32 vio: Add the OPEN_REPARSE_POINTS flag to the CreateFileW call.
Otherwise win32 api simply does not open reparse points at all,
such as junctions.

This will improve the user experience of #3813 as we now identify
the junctions as symlinks.... and ignore them properly.
2015-09-25 17:36:41 +02:00
Klaas Freitag
6930cd6687 csync_update: Handle error if current_fs is zero. 2015-09-11 18:12:45 +02:00
Phil Davis
7d191763b5 Minor build etc typos
Just some random stuff I noticed while looking at how this beast is put
together.
2015-09-10 13:20:11 +05:45
Klaas Freitag
df534753b1 csync_update: Handle permission denied as soft error in discovery.
For that, treat the not accessible directory as if it were ignored.

This will fix #3767
2015-09-09 14:12:13 +02:00
Klaas Freitag
02c077e3f8 vio_local_win: Properly set errno in case of error happened.
Without a proper errno value, the calling function can not detect
the kind of problem.
2015-09-09 14:10:23 +02:00
Christian Kamm
c832a9eee5 csync io: Fix UNC path conversion on Win #3748
Paths that were already in UNC form don't need to be prefixed.
2015-09-04 15:57:21 +02:00
Jocelyn Turcotte
35318ea9b5 Fix the Windows build
Use an iterator variable declared outside of the loop condition
expression.

MinGW doesn't support C99 very easily and it's easier to keep
our C code C89 compliant.
2015-08-31 17:23:25 +02:00
Christian Kamm
07ca0be3c5 csync_excluded: Another speedup #3638
Build a list of path components outside of the exclude pattern loop.
2015-08-24 15:08:06 +02:00
Christian Kamm
e71cca128d csync_update: Use excluded_traversal() to improve performance #3638 2015-08-24 15:08:06 +02:00
Christian Kamm
fe49255bff csync_excluded: Add fast _traversal() function #3638
This function only checks the full path and the basename and is thus
around 7x faster. It is very useful in a csync_update context where
we know that the leading dirs have already been checked for exclusion.
2015-08-24 15:08:06 +02:00
Christian Kamm
20f4ec6b67 csync_exclude: Speed up siginificantly #3638
The performance test indicates a speedup of around 3x for paths
with around 12 components.
2015-08-24 15:08:06 +02:00
Olivier Goffart
9dcce01e54 csync: ignore files/folder for which stat fails
instead of pretending it suicceed and not recursing in it.

This fixes a bug in which a folder with a too long name would be properly
created, then removed on the server in the next sync.

(cherry picked from commit 4bbf7669091cde7ec726b1708d8c54427b68f016)

Conflicts:
	csync/src/csync.h
	csync/src/csync_exclude.h
	csync/src/csync_update.c
2015-08-11 11:17:20 +02:00
Olivier Goffart
2a4396fbdb Fixup ce5be89a18 2015-08-07 11:05:04 +02:00
Olivier Goffart
ce5be89a18 Fix warning
csync_vio_local_unix.c:109:7: warning: ignoring return value of function declared with
      warn_unused_result attribute [-Wunused-result]
      asprintf(&file_stat->original_name, "%s/%s", handle->path, dirent->d_name);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2015-08-06 13:58:04 +02:00
Olivier Goffart
499e13377d check_vio_ext.c: assert_string_equal is a macro and we cannot have #ifdef in it 2015-08-06 10:02:12 +02:00
Olivier Goffart
157f59d44a Csync tests: fix on linux
Linux do not iconv to decode filenames
2015-08-05 11:09:27 +02:00
Markus Goetz
f6cd2c79d5 CSync: Proper error for invalid filenames #2777 #3128 #2240 2015-08-04 22:13:38 +02:00
Markus Goetz
a7b0f7b42e check_vio_ext: Fix on OS X 2015-08-04 17:25:48 +02:00
Markus Goetz
cdb556e404 Reconcile: Improve debug output 2015-07-31 16:43:47 +02:00
Klaas Freitag
0c5528806d csync cmake: Removed not needed define of DATADIR. 2015-07-30 16:38:30 +02:00
Klaas Freitag
3b510969a2 Fix test, make it independent on the sort order of readdir. 2015-07-27 13:28:21 +02:00
Olivier Goffart
0ccaae9a22 Confirm feature: Confirm for any folder, not only shared ones
issue #3148
2015-07-27 09:54:20 +02:00
Klaas Freitag
c72b6fb8e6 csync_update: Reorg the code a bit to make it easier to read and understand 2015-07-15 15:54:41 +02:00
Klaas Freitag
ddf7419aca csync: Initialize the hidden files flag properly. 2015-07-15 15:54:41 +02:00
Klaas Freitag
27e0f9d132 csync: Updated and added some documentation. 2015-07-15 15:54:41 +02:00
Klaas Freitag
a4336092f6 Hidden Files: Add a setting to ignore hidden files or not.
This setting goes to the folder definition. By default, no hidden
files are synced.
2015-07-15 15:54:41 +02:00
Klaas Freitag
e11f59af32 csync_update: store the remote ignore information into tree (from tmp)
This way the propagator is able to skip a remove if ingored files are
exiting.
2015-07-15 15:49:27 +02:00
Klaas Freitag
b0aedef4b0 Sync: Store information about ignored files on the server in database.
That allows better handling of remove requests that should not be done
because the directory is not empty on the remote side (as it contains
ignored files).
2015-07-15 15:49:27 +02:00
Klaas Freitag
c940762b7c csync_update: set has_ignored_files in case of hidden files.
Re-arragned the code a bit for better overview on whats happening.
2015-07-15 15:20:46 +02:00
Klaas Freitag
7aa2b50828 csync_update: Handling hidden files as excluded files, as intended. 2015-07-15 15:20:46 +02:00
Klaas Freitag
e0abbc0b83 vio_local: Properly detect if files are hidden.
On Linux and Mac, check for the leading dot. On Windows,
there is an flag set in the result struct of fetFileInformationByHandle.
2015-07-15 15:20:46 +02:00
Markus Goetz
cd64d3ae2c CSync: Split csync_vio_local.c
This is to make room for further improvements #880
2015-07-13 15:54:09 +02:00
Olivier Goffart
45180a024d Rename should_update_etag to should_update_metadata
This should avoid some confusion
2015-07-09 15:05:37 +02:00
Klaas Freitag
7f628e6f7e c_path: Fix function description. 2015-07-08 18:13:41 +02:00
Klaas Freitag
a076a170ee vio_local: strncat only needs two chars as limit. 2015-07-08 18:13:41 +02:00
Klaas Freitag
ccc91a52a8 vio_local: Give strncpy and strncat enough space to copy closing zero. 2015-07-08 18:13:41 +02:00
Klaas Freitag
4cb78999d2 vio_local: Simplify and fix the error out behaviour of win32 impl.
This avoids leaking of a stat struct in error case.
2015-07-08 18:13:41 +02:00
Klaas Freitag
1a69e2ae8d vio_local: Remove commented code. 2015-07-08 18:13:41 +02:00
Klaas Freitag
c7ba7c8e9f Fix comment for c_path_to_UNC, plus cleanup of superflous debug. 2015-07-08 18:13:41 +02:00
Klaas Freitag
8bb1a49aaf Use strncat and strncpy rather than the strcpy and friends. 2015-07-08 18:13:40 +02:00
Klaas Freitag
d303df7fa0 do not use the path function here. 2015-07-08 18:13:40 +02:00
Klaas Freitag
a280007dc2 Added another test case with content in the deep path (also win32). 2015-07-08 18:13:40 +02:00
Klaas Freitag
d9b44b3a69 Created two functions c_utf8_string_to_locale and c_utf8_path_to_locale.
Before we only had c_utf8_to_locale, but now functionality is needed to
convert a path to UNC before converting it. That does c_utf8_path_to_locale
now, while c_utf8_string_to_locale only converts the plain string, ie.
to generate wide char strings for output.
2015-07-08 18:13:39 +02:00
Klaas Freitag
5214189eb2 Moved the makeWinLongPath function to c_path and rename to c_path_to_UNC
Other cleanups like streamlined allocation and more.
2015-07-08 18:13:39 +02:00
Klaas Freitag
3aed7fc0be Removed dead code from csync_utils. Functionality moved to Qt based code. 2015-07-08 18:13:39 +02:00
Klaas Freitag
2e04eba74b Long Win Pathes: Added test to test dir traversal with long pathes. 2015-07-08 18:13:39 +02:00
Klaas Freitag
2cce8065ab CMocka Testing: Install the cmocka library 2015-07-08 18:13:39 +02:00
Klaas Freitag
8183c37dfe long win pathes: Some minor refinements and documentations 2015-07-08 18:13:39 +02:00
Klaas Freitag
f98af43d31 Add cross platform definition for chdir and getcwd. 2015-07-08 18:13:39 +02:00
Klaas Freitag
b9b5e1cf33 Long win pathes: Make makeLongWinPath function return the num of alloc bytes.
This enables the calling function to free these accordingly. That is needed
because the makeLongWinPath for efficiency reasons does not always realloc
the original string.
2015-07-08 18:13:39 +02:00
Klaas Freitag
2b391396c6 vio local: Use win32 native functions to traverse the file tree.
Native functions to replace the not native readdir and opendir functions.
This works with long filenames now.
2015-07-08 18:13:39 +02:00
Klaas Freitag
5caff8cc93 Add a FIXME comment. 2015-07-08 18:13:39 +02:00