This prevents having to define a Problem structure with dubious
operator overloads to accomplish the same.
Also use std::map::lower_bound to quickly iterate over the
list of problems.
This also remove all smartness from the SocketApi about the status
of a file and solely use info from the current and last sync.
This simplifies the logic a lot and prevents any discrepancy between
the status shown in the activity log and the one displayed on the
overlay icon of a file.
The main benefit of the additional simplicity is that we are able
to push all new status of a file reliably (including warnings for
parent folders) to properly update the icon on overlay implementations
that don't allow us invalidating the status cache, like on OS X.
Both errors and warning from the last sync are now kept in a set,
which is used to also affect parent folders of an error.
To make sure that errors don't become warning icons on a second
sync, SyncFileItem::_hasBlacklistEntry is also interpreted as an error.
This also renames StatusIgnore to StatusWarning to match this semantic.
SyncEngine::aboutToPropagate is used in favor of SyncEngine::syncItemDiscovered
since the latter is emitted before file permission warnings are set on the
SyncFileItem. SyncEngine::finished is not used since we have all the
needed information in SyncEngine::itemCompleted.
SyncFileStatus' purpose is to track overlay icon status.
Instead of putting comments and default: clauses in switch
on both sides about unused enums, use different enums.
This also remove STATUS_NEW which is the equivalent of
STATUS_SYNC in all shell extension implementations, and
remove STATUS_UPDATED and STATUS_STAT_ERROR which have
the same semantic as STATUS_UPTODATE and STATUS__ERROR.
This will help moving the SyncEngine construction in the constructor
and allow moving functionalities from Folder to SyncEngine or its
delegated objects.
Soldiering on with a broken or incomplete response could lead to
incorrect sync behavior.
Since discovery uses LsCol jobs which already handle errors
correctly, this should not have a significant impact.
the size on the server might be different from the size on the client
with certain backend so it should be ignored.
(cherry picked from commit 9222db6df9b19a21e1bea5a238d745d96a6385e3)
In SQLite bindings are not cleared by sqlite3_reset() calls, so
skipping a sqlite3_bind call to create a NULL value doesn't work,
instead the previous value will be written.
To fix this, I clear all bindings in SqlQuery::reset and make sure
to explicitly bind NULL when desired in SqlQuery::bind.
To make sure there's no confusion about SqlQuery::reset and
sqlite3_reset, I rename our method to reset_and_clear_bindings().
(cherry picked from commit 7bd4f95b8c)
In SQLite bindings are not cleared by sqlite3_reset() calls, so
skipping a sqlite3_bind call to create a NULL value doesn't work,
instead the previous value will be written.
To fix this, I clear all bindings in SqlQuery::reset and make sure
to explicitly bind NULL when desired in SqlQuery::bind.
To make sure there's no confusion about SqlQuery::reset and
sqlite3_reset, I rename our method to reset_and_clear_bindings().
The isValid check should be used everywhere the capabilities
are used as the loading of the capabilities is happening
in parallel of the startup, so it is not guaranteed to be
available always.
Helps with small file sync #331
When I benchmarked this, it went up to 6 parallelism and
was about 1/3 faster than the previous fixed 3 parallelism.
Doing more than 6 is dangerous because QNAM limits to 6 TCP
connections and also the server might become a bottleneck.
Should also help for #4081
This will be useful if we ever want to store account-level gui state.
I built this originally because I thought a paused account would be
this kind of state.
The creation doesn't need to be separated from the SyncEngine anymore.
This allows the SyncEngine to be created in fewer steps if we want to
use it in tests.
This moves most of the direct csync code from Folder into the SyncEngine.
The exclude file logic for the context has been wrapped using the
existing ExcludedFiles class as well.
As discussed with Klaas, this seems to be a better compromise.
10MB * 3 prarralel jobs = 30MB in memory, and to retry in case of
disconnection. Which is still reasonable. And might make the upload
almost twice as fast on fast network where the amount of chunk is the
bottleneck (because of more server processing)
Relates to issue #4354
If the PROPFIND return an invalid code (like 200) then we would
not recieve the error signal and we would never sync again.
Found while investigating https://github.com/owncloud/enterprise/issues/1068
The ".sys.admin#recall#" is the recall file and should not be ignored
even if hidden.
The remote discovery do not need to detect hidden files because it
is already detected by csync in all cases. So this avoid code duplication
Users have complained that they don't see the notification when it is
shown and are not aware that their files aren't syncing.
Remove the non-interactive credentials fetch logic and add make sure
that the shibboleth popup will flash in the taskbar instead.
This will still not allow the popup to show in front in all cases,
but this is a compromise that we have to chose.
This reverts commit dcb687929f.
Issue https://github.com/owncloud/enterprise/issues/990
This is the fix for issue #4370
Step to reproduce the bug:
1) have lots of files in directory "dir1"
2) do mkdir dir2 && mv dir1/* dir2
3) DURING the sync (which takes time because of the many moves) do mkdir dir3 && mv dir2/* dir3/
4) observe that files are PUT in the next sync
The problem is that SyncJournalFileRecord::SyncJournalFileRecord will fail to
get the inode after the forst move because the files are already moved on the
filesystem. Normaly it should use the inode from the discovery phase in that
case but that is not working because it comes from the remote node in case of
moves, so the code in SyncEngine::treewalkFile would not set the inode.
Test in https://github.com/owncloud/smashbox/pull/143
Added a new "chunkSize" entry in the General group of the owncloud.cfg
which can be set to the size, in bytes, of the chunks.
This allow user with hude bandwidth to select more optimal chunk size
Issue #4354
Server older than 8.1 cannot cope with invalid char in the filename
so we must not send them from the client. We were already checking
for new files, but not for renames or new directories.
https://github.com/owncloud/enterprise/issues/1009
Previously we would fail to start if the directory was not existing.
This was working for relative directory, but it should also work for
absolute ones
https://github.com/owncloud/enterprise/issues/970
* Ensure every time a file becomes a directory or the other way around
the item is flagged as INSTRUCTION_TYPE_CHANGE.
* Delete the badly-typed entity if necessary in the propagation jobs.
The Qt HTTP thread calls authenticationRequired (indirectly) using a
BlockingQueuedConnection. So when we call invalidateToken from slot
connected to this signal and end up calling QNAM::clearAccessCache which
waits on the thread for 5 seconds
Backtraces:
Qt HTTP thread:
#0 0x00007ffff20c707f in pthread_cond_wait@@GLIBC_2.3.2 ()
#1 0x00007ffff43f0c0b in QWaitConditionPrivate::wait
#2 QWaitCondition::wait
#3 0x00007ffff43ea06b in QSemaphore::acquire
#4 0x00007ffff45dcf6f in QMetaObject::activate
[...]
#9 0x00007ffff45dd607 in QMetaObject::activate
#10 0x00007ffff4edbaf7 in QHttpNetworkReply::authenticationRequired
#11 0x00007ffff4e0b2b4 in QHttpNetworkConnectionPrivate::handleAuthenticateChallenge
#12 0x00007ffff4e10753 in QHttpNetworkConnectionChannel::handleStatus
#13 0x00007ffff4e11cc9 in QHttpNetworkConnectionChannel::allDone
#14 0x00007ffff4e14605 in QHttpProtocolHandler::_q_receiveReply
Main Thread:
#0 0x00007ffff20c7428 in pthread_cond_timedwait@@GLIBC_2.3.2 ()
#1 0x00007ffff43f0b56 in QWaitConditionPrivate::wait_relative (time=5000, this=0x136c580)
#2 QWaitConditionPrivate::wait (time=5000, this=0x136c580)
#3 QWaitCondition::wait (this=this@entry=0x136c788, mutex=mutex@entry=0x136c760, time=time@entry=5000)
#4 0x00007ffff43efa6e in QThread::wait (this=<optimized out>, time=time@entry=5000)
#5 0x00007ffff4e1edd3 in QNetworkAccessManagerPrivate::clearCache
#6 0x00007ffff7b6fb03 in OCC::HttpCredentials::invalidateToken()
#7 0x000000000057adb4 in OCC::AccountState::slotInvalidCredentials()
#8 0x000000000057ac76 in OCC::AccountState::slotConnectionValidatorResult(OCC::ConnectionValidator::Status, QStringList const&) ()
#9 0x00000000005ab45c in OCC::AccountState::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)
#10 0x00007ffff45dcd30 in QMetaObject::activate
#11 0x00007ffff7b78671 in OCC::ConnectionValidator::connectionResult(OCC::ConnectionValidator::Status, QStringList) ()
#12 0x00007ffff7ae2514 in OCC::ConnectionValidator::reportResult(OCC::ConnectionValidator::Status) ()
#13 0x00007ffff7ae39b7 in OCC::ConnectionValidator::slotAuthFailed(QNetworkReply*) ()
#14 0x00007ffff7b784a9 in OCC::ConnectionValidator::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#15 0x00007ffff45dcd30 in QMetaObject::activate
#16 0x00007ffff7b766dc in OCC::AbstractNetworkJob::networkError(QNetworkReply*)
#17 0x00007ffff7af9f6e in OCC::AbstractNetworkJob::slotFinished()
#18 0x00007ffff7b7654d in OCC::AbstractNetworkJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#20 0x00007ffff45dd607 in QMetaObject::activate
#21 0x00007ffff4edd143 in QNetworkReply::finished
#22 0x00007ffff4e3fec7 in QNetworkReplyHttpImplPrivate::finished
#23 0x00007ffff4e41818 in QNetworkReplyHttpImpl::close
#24 0x00007ffff7b7047b in OCC::HttpCredentials::slotAuthentication(QNetworkReply*, QAuthenticator*) ()
#25 0x00007ffff7b79092 in OCC::HttpCredentials::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#27 0x00007ffff45dd607 in QMetaObject::activate
#28 0x00007ffff4e1d6fb in QNetworkAccessManager::authenticationRequired
#29 0x00007ffff4e1ea07 in QNetworkAccessManagerPrivate::authenticationRequired
#30 0x00007ffff4e3c784 in QNetworkReplyHttpImplPrivate::httpAuthenticationRequired
Another case of Main Thread:
#5 0x00007ffff4e1edd3 in QNetworkAccessManagerPrivate::clearCache
#6 0x00007ffff7b6fb03 in OCC::HttpCredentials::invalidateToken()
#7 0x000000000057b1e4 in OCC::AccountState::slotInvalidCredentials() ()
#8 0x00000000005abb8a in OCC::AccountState::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#9 0x00007ffff45dcd30 in QMetaObject::activate
#10 0x00007ffff7b76ed5 in OCC::Account::invalidCredentials() ()
#11 0x00007ffff7ad55f5 in OCC::Account::handleInvalidCredentials()
#12 0x00007ffff7afa69a in OCC::AbstractNetworkJob::slotFinished()
#13 0x00007ffff7b7654d in OCC::AbstractNetworkJob::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#15 0x00007ffff45dd607 in QMetaObject::activate
#16 0x00007ffff4edd143 in QNetworkReply::finished
#17 0x00007ffff4e3fec7 in QNetworkReplyHttpImplPrivate::finished
#18 0x00007ffff4e41818 in QNetworkReplyHttpImpl::close
#19 0x00007ffff7b7047b in OCC::HttpCredentials::slotAuthentication(QNetworkReply*, QAuthenticator*) ()
#20 0x00007ffff7b79092 in OCC::HttpCredentials::qt_static_metacall(QObject*, QMetaObject::Call, int, void**) ()
#22 0x00007ffff45dd607 in QMetaObject::activate
#23 0x00007ffff4e1d6fb in QNetworkAccessManager::authenticationRequired
The error status of children should only be used for the etag logic.
The SocketApi uses a path matching system to do this and the UI should
report errors only for individual involved files/directories.
Since the presence of any path in SyncEngine::_syncedItems
would translate in a SYNC status, platforms that don't refresh
all their status cache after an UPDATE_VIEW message like OS X
or Windows would keep displaying that status even after all
files are successfully synchronized.
- Read SyncFileItem::_status to determine the status to display mid-sync
- Match moved paths also to _renameTarget since this might be the
path to match
- Make sure that PropagateDirectory jobs also set SyncFileItem::_status
properly
If all the files bring us to past timestamp, it is possibly a backup
restoration in the server. In which case we want don't want to just
overwrite newer files with the older ones.
Issue #2325
Since owncloud 2.1, csync_vio_local_stat was optimized because readdir
would already fetch most of the info. This works for the discovery,
but not later. And we used this function later for symliks and co.
So this fixes the .lnk on windows
Issue #4300
