Use a QMap to avoid using a full hashtable for only a few entries, and
clear the QMap once we're done with the measuring. This saves a few
hundred bytes per job during propagation that would otherwise only be
freed at the end of the sync.
During propagation, we create a line for each file, taking memory, but
we delete all lines passed 2000 right at the beginning of the next sync.
Since the user has little chances of being able to read past those 2000
lines in the log, we might as well keep it capped at 2000 also during
propagation to prevent it from eating memory.
The SyncRunFileLog owned by the Folder must be destroyed after the
SyncEngine since the SyncEngine will abort during destruction, resulting
in all jobs being aborted.
It's possible that this crash only happens with a debug build.
The FolderWatcher inserts files to be marked as SYNC and we
currently assume that all file statuses will be updated by the
following sync. It's however possible that the FolderWatcher
notify us of a change that csync won't consider necessary to
propagate, in which case a new status wouldn't be pushed and
the file manager would continue showing this file as syncing.
Re-push the file status when emptying the dirty files list
before propagating to avoid this issue, most likely the OK
status.
No need to allocate (and initialize to 0) a 10 MiB buffer for each files, even
when most files are much smaller than that.
So make sure the buffer that we allocate is not bigger than the file size.
And Also 10 MiB is a bit big for a buffer. 500 KiB should be more than enough.
(Too big allocations can cause problem because of memory fragmentation and such)
We first need to set the abort flag to csync and then aborting the discovery
job, otherwise, the discovery thread could start a new job in the mean time.
We also need to make sure that the thread has existed before we destroy the
exclude list.
Events from the crash reporter suggest that the QNAM and its
child replies might get deleted before returning from this method
and the only possible cause we can see is that the inner event
loop has something to do with it.
Try keeping a ref on the QNAM while in this method to make sure
that it won't get deleted by the inner event loop.
Same fix as in commit 60c101d9
From the crash reporter:
Crash
EXCEPTION_ACCESS_VIOLATION_READ at 0x4
qnetworkreply.cpp in QNetworkReply::request at line 476
propagateupload.cpp in OCC::PUTFileJob::slotTimeout at line 100
moc_abstractnetworkjob.cpp in OCC::AbstractNetworkJob::qt_static_metacall at line 98
qobject.cpp in QMetaObject::activate at line 3716
moc_qtimer.cpp in QTimer::timeout at line 192
qtimer.cpp in QTimer::timerEvent at line 247
qobject.cpp in QObject::event at line 1267
qapplication.cpp in QApplicationPrivate::notify_helper at line 3722
qapplication.cpp in QApplication::notify at line 3505
qcoreapplication.cpp in QCoreApplication::notifyInternal at line 932
FolderDefinition::save and load escapes the alias. We also need to escape
it when we remove it.
New folder can't be created with alias that needs escaping, but old folder
from old config may still exist, and we must allow user to delete them.
For issue #4927:
On Windows 10, we get a notification after the sync is finished for file that were
just downloaded. The guard we have against our "own changes" are only working when
the sync is running and the OwncloudPropagator still alive.
Setting the Environment variable only for owncloud makes in inconsistant with
other Qt application running at the same time.
The users can still set it themself for the whole desktop if they wish.
Addresses #4840
Previously rejecting any kind of certificate meant that the user
was never asked again, even if the certificate changed.
Now we keep track of which certificates were rejected and ask again
if the ones mentioned in the ssl errors change.
mitmproxy is excellent for testing this.
* Progress: Don't count dirs without propagation jobs #4856
These directory SyncFileItems are necessary for bookkeeping
but should not influence the progress display at all.
* Progress: Skip ignored files #4856
The problem in this case is if we rename the file "xxx" to "invalid\file".
The rename will fail because the new filename constains a slash, and it
will be blacklisted.
But then if the user re-rename the file to "valid_name", then we should
invalidate the blacklist entry and retry to upload. But we did not do
that because renaming don't change the mtime and we did not store the
rename target in the database
IL issue 558
This fixes an issue in which too many jobs are started un parallel
while uploading many files, which could cause too much memory usage as the
chunks are stored in memory.
Probably the fix for #4611
Issue #4855
A typo in the context string made the translation lookup fail.
But also the %Ln was not recognized as a plural form by transifex, so only
the singular was translated
The assert was there to make sure that this case wasn't happening
to eventually be properly tested. Remove the assert for now but this
codepath should eventually be unit tested using this specific situation.
Since the windows implementation first does cache lookups using the
path string, directories need to be passed identically as through
RETRIEVE_FILE_STATUS.
Change the convention to never have a trailing slash for directories
in the protocol. This allows the convention to be applied without
having to access the disk (since we'd need to know if the path is
represented by a directory) and also matches the convention of the
rest of the sync engine. Individual file manager plugins are then
responsible of handling pushed paths as not ending with a trailing
slash.
This also:
- Moves the trailing slash removal logic from the SyncFileStatusTracker
to the SocketApi class
- Remove the unneeded QString::normalized call in fileStatus, since
this should already be done by the FolderWatcher and plugins
Since the statuses are cached and that we can't invalidate the cache,
sending NOP would need to be overwritten by the default OK status
once the client successfully connected. But instead of remembering
which files we NOPed, rather wait until we are ready to sync before
sending the REGISTER_PATH message to the socket API client. It will
also prevent the client from sending unnecessary RETRIEVE_FILE_STATUS
requests.
Also remove AccountState::canSync, since it does the same as
isConnected and syncing is not an account responsibility.
Go through fileStatus like other cases to make sure that all use
cases go through the same code path. This also makes sure to use
lookupProblem which will use lower_bound which is more efficient
for larger sets of sync problems.
This also fixes the issue with lookupProblem that prevented it to
properly match an empty pathToMatch, caused by the fact that the
problem map contains relative paths not starting with a slash.
Make sure that we push the new status when the status of the SyncEngine
changed. SyncEngine::started comes a bit late, only when the propagation
starts, although it's better in this case since child folders will
only switch to Sync in aboutToPropagate.
Also fix an issue with SyncEngine::findSyncItem when using an empty
fileName; this would match and return the wrong item, even though
not currently happening with the code since fileStatus won't call
it with an empty fileName anymore.
As before, we rely on metadata-update SyncFileItem entries for parent
directories to notify us that a directory contains files to propagate,
and to know when all children were propagated through its itemCompleted
signal.
Those metadata SyncFileItems however have a None direction and we need
to add a explicit directory check to show them as Sync.
This fix also handles new files as well as existing ones, so no need
to keep a separate logic for new files.
propagatedownload.cpp:712:35: error: 'seenLockedFile' is a protected member of 'OCC::OwncloudPropagator'
Signals are protected in Qt4 but public in Qt5, mark the class accessing it
as friend when compiling with Qt4
Do the visual stuff from designer.
The previous code that was ment to change the color in red did not work
and changed it to gray instead.
Also I don't see why there should be a frame.
Issue #4773
When a conflict-rename or a temporary-rename fails, notify the
LockWatcher. It'll regularly check whether the file has become
accesible again. When it has, another sync is triggered.
owncloud/enterprise#1288
If the downloaded file is empty but the PROPFIND previously announced it
should not have been empty, this might mean the file was somehow corrupted
because of a bug on the server and that we should therefore not accept
the file.
Normaly we accept a change between the actual size of the file and what we
got during discovery because the file might have been updated to a new version
inbetween. But after this patch we won't accept the file if it was replaced
by an empty file.
Will help for issue #4583
Also requested by IL for issue 548
This uses the file watcher to keep track of files that were modified
in order to assign them the blue icon.
This is transient state that's not persisted across restarts.
Before commit 1a51b6718a, the wizard was
making sure folder had an alias but this is no longer the case.
So generate still an unique alias.
Alias is not used in the UI any longer, it's just use for internal purposes.
For issue #4737
As discussed on issue ##4460
Having the quote to be queried on subfolder is wrong in the generic case,
so add a branding option to configure it.
This partially reverts commit ff4cdc3161
In the before-propagate slot, new files that wait to be
pushed to the server are remembered in the _syncProblems
map. That way, the parents show a sync icon properly as
asked for in #4682.
After the item has been transfered properly, the item is
removed from the map again because success is the default.
Added in previous commit from pull request #4663
As discussed, we do not need this option so no need to introduce
a new dependency on the config file in the sync engine
If an app modifies the expiration date (for example the password policy
app) then on more recent versions of the server we will get the share
object back REST style. We should use that info!
Fixes#4409
* Add checksums/supportedTypes and checksums/preferredUploadType
capabilities. The default is that no checksum types are supported.
* Remove the transmissionChecksum config option. Servers must now
use the capabilities to indicate that they are fine with the
client sending checksums.
Note: This intentionally breaks brandings that overrode
Theme::transmissionChecksum. The override must be removed and the
server's capabilities must be adjusted to include the new values.
The reason is that updateFolderView is invoked by the
emitted signal folderSyncStateChange() anyway.
This will reduce the traffic over the SocketAPI nicely,
maybe this was the reason why it was slower than before.
There were two issues:
* With the refactoring of how Folder and SyncEngine relate, the
ignore_hidden_files flag on the CSync context was reset after
each sync run and not updated from the configuration again.
* The folder watcher failed to enumerate hidden folders and thus
didn't watch for changes inside them. (linux only)
SQLITE_DONE is the indicator for not more query results, which is a legal
thing and not an error condition.
Also, check _getFileRecordQuery for null pointer, as close() wipes it.
The problem with QSet is that the QDateTime was part of
the hash, but that does not make sens as it should be unique
per widget and not per <date, widget>
Instead make it a QHash so there is only one entry per widget.
The idea is that the next call to any database operation will try to
reopen the database through the checkConnect() method. So even if there
was a disconnect trom the db file, this will reestablish the connection.
Imagine tgus scenario on a read only share that you move file from
one location to a new directory in the read only share.
Creating the read only directory fails for permission error.
But we should also restore the files that have been moved.
IL issue 542
Bring back the hardcoded status logic for excluded files.
Since the activity log doesn't even mention those files on purpose,
we can't rely on the SyncEngine to notify us about the status of those files.
Looking up a/aa while an error is present in a/aab/aaba would return
a warning status since a/aa is a substring of a/aab.
Fix the issue by checking if the following character is a slash.
This prevents having to define a Problem structure with dubious
operator overloads to accomplish the same.
Also use std::map::lower_bound to quickly iterate over the
list of problems.
* Remove duplicate remote path
* Use thin progress bar
* Move bandwidth and file info to tooltip
* Shorten overall progress message
This also fixes#4562 by making the layout not dependent on the
width of the displayed text.
This also remove all smartness from the SocketApi about the status
of a file and solely use info from the current and last sync.
This simplifies the logic a lot and prevents any discrepancy between
the status shown in the activity log and the one displayed on the
overlay icon of a file.
The main benefit of the additional simplicity is that we are able
to push all new status of a file reliably (including warnings for
parent folders) to properly update the icon on overlay implementations
that don't allow us invalidating the status cache, like on OS X.
Both errors and warning from the last sync are now kept in a set,
which is used to also affect parent folders of an error.
To make sure that errors don't become warning icons on a second
sync, SyncFileItem::_hasBlacklistEntry is also interpreted as an error.
This also renames StatusIgnore to StatusWarning to match this semantic.
SyncEngine::aboutToPropagate is used in favor of SyncEngine::syncItemDiscovered
since the latter is emitted before file permission warnings are set on the
SyncFileItem. SyncEngine::finished is not used since we have all the
needed information in SyncEngine::itemCompleted.
SyncFileStatus' purpose is to track overlay icon status.
Instead of putting comments and default: clauses in switch
on both sides about unused enums, use different enums.
This also remove STATUS_NEW which is the equivalent of
STATUS_SYNC in all shell extension implementations, and
remove STATUS_UPDATED and STATUS_STAT_ERROR which have
the same semantic as STATUS_UPTODATE and STATUS__ERROR.
This currently is no-op code since the socket API isn't notified
that the tainted folder list changed, and the result is the same
since a sync will be triggered within the next 5 seconds and the
modified folder will be shown as SYNC at that point anyway.
Removing the dependency to the file watcher allows moving the
status estimation logic to libsync.
This will help moving the SyncEngine construction in the constructor
and allow moving functionalities from Folder to SyncEngine or its
delegated objects.
The policy that was said is that if a notification has no action, the
client can and should display a close-button. This patch does it.
In additon to that, the client needs a blacklist of closed notifcations
otherwise they would re-appear next time the server notifications are
fetched again.
Also, changed the cleanup of not-longer-used widgets to be more robust.
If a notification is not longer in the list that comes from the
server, the notification is removed.
That is mainly for the notifications that are created by the
announcement application
Since the quota is a per-folder value, this will make the displayed data
more useful when a single sync folder is configured.
Of course each subfolder could have a different quota again.
The date we receive from the server is an ISO8601 datetime that
includes the offset from UTC. Qt does correctly parse this
information and creates the appropriate QDateTime object.
Calling setTimeSpec(UTC) will force the timezone offset to 0 and
thereby change the referenced point in time to an incorrect one.
Soldiering on with a broken or incomplete response could lead to
incorrect sync behavior.
Since discovery uses LsCol jobs which already handle errors
correctly, this should not have a significant impact.
