mirrors/nextcloud-desktop
1
0
Fork 0
mirror of https://github.com/nextcloud/desktop.git synced 2024-10-23 12:55:44 +03:00
Code Issues Projects Releases Packages Wiki Activity
nextcloud-desktop/test/testsyncengine.cpp

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

620 lines
27 KiB
C++
Raw Normal View History

Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
/*
* This software is in the public domain, furnished "as is", without technical
* support, and with no warranty, express or implied, as to its usefulness for
* any purpose.
*
*/
#include <QtTest>
#include "syncenginetestutils.h"
SyncEngine: Fix renaming a folder should keep the selective sync state Issue #5224 Two problems: - In the discovery phase, we need to check the selective sync entries of the source path in case of renames. - When the rename is done, we need to actually update the black list in the database.
2016-10-10 17:55:31 +03:00
#include <syncengine.h>
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
using namespace OCC;
bool itemDidComplete(const QSignalSpy &spy, const QString &path)
{
for(const QList<QVariant> &args : spy) {
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
auto item = args[0].value<SyncFileItemPtr>();
if (item->destination() == path)
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
return true;
}
return false;
}
bool itemDidCompleteSuccessfully(const QSignalSpy &spy, const QString &path)
{
for(const QList<QVariant> &args : spy) {
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
auto item = args[0].value<SyncFileItemPtr>();
if (item->destination() == path)
return item->_status == SyncFileItem::Success;
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
}
return false;
}
class TestSyncEngine : public QObject
{
Q_OBJECT
private slots:
void testFileDownload() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.remoteModifier().insert("A/a0");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "A/a0"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testFileUpload() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.localModifier().insert("A/a0");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "A/a0"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testDirDownload() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.remoteModifier().mkdir("Y");
fakeFolder.remoteModifier().mkdir("Z");
fakeFolder.remoteModifier().insert("Z/d0");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Y"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Z"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Z/d0"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testDirUpload() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.localModifier().mkdir("Y");
fakeFolder.localModifier().mkdir("Z");
fakeFolder.localModifier().insert("Z/d0");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Y"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Z"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "Z/d0"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testLocalDelete() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.remoteModifier().remove("A/a1");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "A/a1"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testRemoteDelete() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.localModifier().remove("A/a1");
fakeFolder.syncOnce();
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "A/a1"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
void testEmlLocalChecksum() {
FakeFolder fakeFolder{FileInfo{}};
fakeFolder.localModifier().insert("a1.eml", 64, 'A');
fakeFolder.localModifier().insert("a2.eml", 64, 'A');
fakeFolder.localModifier().insert("a3.eml", 64, 'A');
Checksums: Clearer behavior and added testing
2017-09-15 14:30:29 +03:00
fakeFolder.localModifier().insert("b3.txt", 64, 'A');
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
// Upload and calculate the checksums
// fakeFolder.syncOnce();
fakeFolder.syncOnce();
Checksums: Clearer behavior and added testing
2017-09-15 14:30:29 +03:00
auto getDbChecksum = [&](QString path) {
SyncJournalDB: Allow callers of getFileRecord if the query failed The current implementation would return the same value whether the query failed or if no row would be found. This is something that is currently checked by csync and needs to be provided if we want to use SyncJournalDB there. Adjusted all call sites to also check the return value even though they could still just rely on rec.isValid(), but makes it more explicit as to what happens for database errors in those cases, if we ever want to gracefully handle them.
2017-09-13 20:02:38 +03:00
SyncJournalFileRecord record;
fakeFolder.syncJournal().getFileRecord(path, &record);
Checksums: Clearer behavior and added testing
2017-09-15 14:30:29 +03:00
return record._checksumHeader;
};
// printf 'A%.0s' {1..64} | sha1sum -
QByteArray referenceChecksum("SHA1:30b86e44e6001403827a62c58b08893e77cf121f");
QCOMPARE(getDbChecksum("a1.eml"), referenceChecksum);
QCOMPARE(getDbChecksum("a2.eml"), referenceChecksum);
QCOMPARE(getDbChecksum("a3.eml"), referenceChecksum);
QCOMPARE(getDbChecksum("b3.txt"), referenceChecksum);
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
// Touch the file without changing the content, shouldn't upload
fakeFolder.localModifier().setContents("a1.eml", 'A');
// Change the content/size
fakeFolder.localModifier().setContents("a2.eml", 'B');
fakeFolder.localModifier().appendByte("a3.eml");
Checksums: Clearer behavior and added testing
2017-09-15 14:30:29 +03:00
fakeFolder.localModifier().appendByte("b3.txt");
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
fakeFolder.syncOnce();
Checksums: Clearer behavior and added testing
2017-09-15 14:30:29 +03:00
QCOMPARE(getDbChecksum("a1.eml"), referenceChecksum);
QCOMPARE(getDbChecksum("a2.eml"), QByteArray("SHA1:84951fc23a4dafd10020ac349da1f5530fa65949"));
QCOMPARE(getDbChecksum("a3.eml"), QByteArray("SHA1:826b7e7a7af8a529ae1c7443c23bf185c0ad440c"));
QCOMPARE(getDbChecksum("b3.eml"), getDbChecksum("a3.txt"));
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
QVERIFY(!itemDidComplete(completeSpy, "a1.eml"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "a2.eml"));
QVERIFY(itemDidCompleteSuccessfully(completeSpy, "a3.eml"));
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
}
SyncEngine: Fix renaming of folder when file are changed (#5195) Two bugs: - The change filed are not considered as move, they are re-downloaded but the old file was not removed from the database. The change in owncloudpropagator.cpp takes care of removing the old entries. - Next sync would then remove the file in the server in the old folder This was not a problem until we start reusing the sync engine, and that the _renamedFolders map is not cleared. We were before deleting a non-existing file. But now we delete the actual file. Also improve the tests to be able to do move on the server. This include support for file id. Issue #5192
2016-09-22 10:02:47 +03:00
void testRemoteChangeInMovedFolder() {
// issue #5192
FakeFolder fakeFolder{FileInfo{ QString(), {
FileInfo { QStringLiteral("folder"), {
FileInfo{ QStringLiteral("folderA"), { { QStringLiteral("file.txt"), 400 } } },
QStringLiteral("folderB")
}
}}}};
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
// Edit a file in a moved directory.
fakeFolder.remoteModifier().setContents("folder/folderA/file.txt", 'a');
fakeFolder.remoteModifier().rename("folder/folderA", "folder/folderB/folderA");
fakeFolder.syncOnce();
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
auto oldState = fakeFolder.currentLocalState();
SyncEngine: Fix renaming a folder should keep the selective sync state Issue #5224 Two problems: - In the discovery phase, we need to check the selective sync entries of the source path in case of renames. - When the rename is done, we need to actually update the black list in the database.
2016-10-10 17:55:31 +03:00
QVERIFY(oldState.find("folder/folderB/folderA/file.txt"));
QVERIFY(!oldState.find("folder/folderA/file.txt"));
SyncEngine: Fix renaming of folder when file are changed (#5195) Two bugs: - The change filed are not considered as move, they are re-downloaded but the old file was not removed from the database. The change in owncloudpropagator.cpp takes care of removing the old entries. - Next sync would then remove the file in the server in the old folder This was not a problem until we start reusing the sync engine, and that the _renamedFolders map is not cleared. We were before deleting a non-existing file. But now we delete the actual file. Also improve the tests to be able to do move on the server. This include support for file id. Issue #5192
2016-09-22 10:02:47 +03:00
// This sync should not remove the file
fakeFolder.syncOnce();
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
QCOMPARE(fakeFolder.currentLocalState(), oldState);
}
SyncEngine: Fix renaming a folder should keep the selective sync state Issue #5224 Two problems: - In the discovery phase, we need to check the selective sync entries of the source path in case of renames. - When the rename is done, we need to actually update the black list in the database.
2016-10-10 17:55:31 +03:00
void testSelectiveSyncModevFolder() {
// issue #5224
FakeFolder fakeFolder{FileInfo{ QString(), {
FileInfo { QStringLiteral("parentFolder"), {
FileInfo{ QStringLiteral("subFolderA"), { { QStringLiteral("fileA.txt"), 400 } } },
FileInfo{ QStringLiteral("subFolderB"), { { QStringLiteral("fileB.txt"), 400 } } }
}
}}}};
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
auto expectedServerState = fakeFolder.currentRemoteState();
// Remove subFolderA with selectiveSync:
fakeFolder.syncEngine().journal()->setSelectiveSyncList(SyncJournalDb::SelectiveSyncBlackList,
{"parentFolder/subFolderA/"});
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
fakeFolder.syncEngine().journal()->avoidReadFromDbOnNextSync(QByteArrayLiteral("parentFolder/subFolderA/"));
SyncEngine: Fix renaming a folder should keep the selective sync state Issue #5224 Two problems: - In the discovery phase, we need to check the selective sync entries of the source path in case of renames. - When the rename is done, we need to actually update the black list in the database.
2016-10-10 17:55:31 +03:00
fakeFolder.syncOnce();
{
// Nothing changed on the server
QCOMPARE(fakeFolder.currentRemoteState(), expectedServerState);
// The local state should not have subFolderA
auto remoteState = fakeFolder.currentRemoteState();
remoteState.remove("parentFolder/subFolderA");
QCOMPARE(fakeFolder.currentLocalState(), remoteState);
}
// Rename parentFolder on the server
fakeFolder.remoteModifier().rename("parentFolder", "parentFolderRenamed");
expectedServerState = fakeFolder.currentRemoteState();
fakeFolder.syncOnce();
{
QCOMPARE(fakeFolder.currentRemoteState(), expectedServerState);
auto remoteState = fakeFolder.currentRemoteState();
// The subFolderA should still be there on the server.
QVERIFY(remoteState.find("parentFolderRenamed/subFolderA/fileA.txt"));
// But not on the client because of the selective sync
remoteState.remove("parentFolderRenamed/subFolderA");
QCOMPARE(fakeFolder.currentLocalState(), remoteState);
}
// Rename it again, locally this time.
fakeFolder.localModifier().rename("parentFolderRenamed", "parentThirdName");
fakeFolder.syncOnce();
{
auto remoteState = fakeFolder.currentRemoteState();
// The subFolderA should still be there on the server.
QVERIFY(remoteState.find("parentThirdName/subFolderA/fileA.txt"));
// But not on the client because of the selective sync
remoteState.remove("parentThirdName/subFolderA");
QCOMPARE(fakeFolder.currentLocalState(), remoteState);
expectedServerState = fakeFolder.currentRemoteState();
Pass the SyncFileItem as SyncFileItemPtr in itemCompleted This will allow us to keep a reference on the items in connected slots.
2017-01-25 13:12:38 +03:00
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
SyncEngine: Fix renaming a folder should keep the selective sync state Issue #5224 Two problems: - In the discovery phase, we need to check the selective sync entries of the source path in case of renames. - When the rename is done, we need to actually update the black list in the database.
2016-10-10 17:55:31 +03:00
fakeFolder.syncOnce(); // This sync should do nothing
QCOMPARE(completeSpy.count(), 0);
QCOMPARE(fakeFolder.currentRemoteState(), expectedServerState);
QCOMPARE(fakeFolder.currentLocalState(), remoteState);
}
}
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
void testSelectiveSyncBug() {
// issue owncloud/enterprise#1965: files from selective-sync ignored
// folders are uploaded anyway is some circumstances.
FakeFolder fakeFolder{FileInfo{ QString(), {
FileInfo { QStringLiteral("parentFolder"), {
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
FileInfo{ QStringLiteral("subFolderA"), {
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
{ QStringLiteral("fileA.txt"), 400 },
SyncEngine: SelectiveSync: Remove local files of undelected folder despite other modified files Issue #5783 When the directry that should be removed by selective sync contains changes, we ignore the whole sub tree instead of only ignoreing new files. We cannot ignore the whole directory, we need to ignore only the directory that do not have files to remove
2017-06-06 17:00:41 +03:00
{ QStringLiteral("fileB.txt"), 400, 'o' },
FileInfo { QStringLiteral("subsubFolder"), {
{ QStringLiteral("fileC.txt"), 400 },
{ QStringLiteral("fileD.txt"), 400, 'o' }
}},
FileInfo{ QStringLiteral("anotherFolder"), {
FileInfo { QStringLiteral("emptyFolder"), { } },
FileInfo { QStringLiteral("subsubFolder"), {
{ QStringLiteral("fileE.txt"), 400 },
{ QStringLiteral("fileF.txt"), 400, 'o' }
}}
}}
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
}},
FileInfo{ QStringLiteral("subFolderB"), {} }
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
}}
}}};
QCOMPARE(fakeFolder.currentLocalState(), fakeFolder.currentRemoteState());
auto expectedServerState = fakeFolder.currentRemoteState();
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
// Remove subFolderA with selectiveSync:
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
fakeFolder.syncEngine().journal()->setSelectiveSyncList(SyncJournalDb::SelectiveSyncBlackList,
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
{"parentFolder/subFolderA/"});
fakeFolder.syncEngine().journal()->avoidReadFromDbOnNextSync(QByteArrayLiteral("parentFolder/subFolderA/"));
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
SyncEngine: SelectiveSync: Remove local files of undelected folder despite other modified files Issue #5783 When the directry that should be removed by selective sync contains changes, we ignore the whole sub tree instead of only ignoreing new files. We cannot ignore the whole directory, we need to ignore only the directory that do not have files to remove
2017-06-06 17:00:41 +03:00
// But touch local file before the next sync, such that the local folder
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
// can't be removed
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
fakeFolder.localModifier().setContents("parentFolder/subFolderA/fileB.txt", 'n');
fakeFolder.localModifier().setContents("parentFolder/subFolderA/subsubFolder/fileD.txt", 'n');
fakeFolder.localModifier().setContents("parentFolder/subFolderA/anotherFolder/subsubFolder/fileF.txt", 'n');
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
// Several follow-up syncs don't change the state at all,
// in particular the remote state doesn't change and fileB.txt
// isn't uploaded.
for (int i = 0; i < 3; ++i) {
fakeFolder.syncOnce();
{
// Nothing changed on the server
QCOMPARE(fakeFolder.currentRemoteState(), expectedServerState);
// The local state should still have subFolderA
auto local = fakeFolder.currentLocalState();
Use SyncJournalDb in csync This gets rid of the csync_statedb sqlite layer and use the same code and same connection as the rest of the SyncEngine. Missing functions are added to SyncJournalDb and change a few minor things (like changing SyncJournalFileRecord::_modtime to be an int64 instead of a QDateTime, like it was in csync).
2017-09-14 16:50:13 +03:00
QVERIFY(local.find("parentFolder/subFolderA"));
QVERIFY(!local.find("parentFolder/subFolderA/fileA.txt"));
QVERIFY(local.find("parentFolder/subFolderA/fileB.txt"));
QVERIFY(!local.find("parentFolder/subFolderA/subsubFolder/fileC.txt"));
QVERIFY(local.find("parentFolder/subFolderA/subsubFolder/fileD.txt"));
QVERIFY(!local.find("parentFolder/subFolderA/anotherFolder/subsubFolder/fileE.txt"));
QVERIFY(local.find("parentFolder/subFolderA/anotherFolder/subsubFolder/fileF.txt"));
QVERIFY(!local.find("parentFolder/subFolderA/anotherFolder/emptyFolder"));
QVERIFY(local.find("parentFolder/subFolderB"));
Selective sync: Skip excluded folders when reading db When a new folder becomes selective-sync excluded, we already mark it and all its parent folders with _invalid_ etags to force rediscovery. That's not enough however. Later calls to csync_statedb_get_below_path could still pull data about the excluded files into the remote tree. That lead to incorrect behavior, such as uploads happening for folders that had been explicitly excluded from sync. To fix the problem, statedb_get_below_path is adjusted to not read the data about excluded folders from the database. Currently we can't wipe this data from the database outright because we need it to determine whether the files in the excluded folder can be wiped away or not. See owncloud/enterprise#1965
2017-05-15 15:46:09 +03:00
}
}
}
Delete finished propagation jobs in PropagateDirectory #5269 (#5400)
2017-01-26 12:03:22 +03:00
void abortAfterFailedMkdir() {
FakeFolder fakeFolder{FileInfo{}};
QSignalSpy finishedSpy(&fakeFolder.syncEngine(), SIGNAL(finished(bool)));
fakeFolder.serverErrorPaths().append("NewFolder");
fakeFolder.localModifier().mkdir("NewFolder");
// This should be aborted and would otherwise fail in FileInfo::create.
fakeFolder.localModifier().insert("NewFolder/NewFile");
fakeFolder.syncOnce();
QCOMPARE(finishedSpy.size(), 1);
QCOMPARE(finishedSpy.first().first().toBool(), false);
}
Propagator: Fix finished signal of directory being emited twice and causing crash When there is a FatalError, we ended up emiting the finished signal for the directory job several times, which would lead to crashes Issue #5578
2017-03-06 15:07:08 +03:00
void testDirDownloadWithError() {
FakeFolder fakeFolder{FileInfo::A12_B12_C12_S12()};
QSignalSpy completeSpy(&fakeFolder.syncEngine(), SIGNAL(itemCompleted(const SyncFileItemPtr &)));
fakeFolder.remoteModifier().mkdir("Y");
fakeFolder.remoteModifier().mkdir("Y/Z");
fakeFolder.remoteModifier().insert("Y/Z/d0");
fakeFolder.remoteModifier().insert("Y/Z/d1");
fakeFolder.remoteModifier().insert("Y/Z/d2");
fakeFolder.remoteModifier().insert("Y/Z/d3");
fakeFolder.remoteModifier().insert("Y/Z/d4");
fakeFolder.remoteModifier().insert("Y/Z/d5");
fakeFolder.remoteModifier().insert("Y/Z/d6");
fakeFolder.remoteModifier().insert("Y/Z/d7");
fakeFolder.remoteModifier().insert("Y/Z/d8");
fakeFolder.remoteModifier().insert("Y/Z/d9");
fakeFolder.serverErrorPaths().append("Y/Z/d2", 503); // 503 is a fatal error
fakeFolder.serverErrorPaths().append("Y/Z/d3", 503); // 503 is a fatal error
QVERIFY(!fakeFolder.syncOnce());
QCoreApplication::processEvents(); // should not crash
QSet<QString> seen;
for(const QList<QVariant> &args : completeSpy) {
auto item = args[0].value<SyncFileItemPtr>();
Remove SyncFileItem::_isDirectory It's always equivalent to _type == SyncFileItem::Directory.
2017-08-24 18:31:46 +03:00
qDebug() << item->_file << item->isDirectory() << item->_status;
Propagator: Fix finished signal of directory being emited twice and causing crash When there is a FatalError, we ended up emiting the finished signal for the directory job several times, which would lead to crashes Issue #5578
2017-03-06 15:07:08 +03:00
QVERIFY(!seen.contains(item->_file)); // signal only sent once per item
seen.insert(item->_file);
Fix TestSyncEngine::testDirDownloadWithError On master, the scheduling is different and the tasks abort in another order
2017-03-09 12:35:58 +03:00
if (item->_file == "Y/Z/d2") {
Propagator: Fix finished signal of directory being emited twice and causing crash When there is a FatalError, we ended up emiting the finished signal for the directory job several times, which would lead to crashes Issue #5578
2017-03-06 15:07:08 +03:00
QVERIFY(item->_status == SyncFileItem::FatalError);
Fix TestSyncEngine::testDirDownloadWithError On master, the scheduling is different and the tasks abort in another order
2017-03-09 12:35:58 +03:00
} else if(item->_file == "Y/Z/d3") {
QVERIFY(item->_status != SyncFileItem::Success);
Propagator: Fix finished signal of directory being emited twice and causing crash When there is a FatalError, we ended up emiting the finished signal for the directory job several times, which would lead to crashes Issue #5578
2017-03-06 15:07:08 +03:00
}
QVERIFY(item->_file != "Y/Z/d9"); // we should have aborted the sync before d9 starts
}
}
Compare the hash of files with identical mtime/size #5589 * For conflicts where mtime and size are identical: a) If there's no remote checksum, skip (unchanged) b) If there's a remote checksum that's a useful hash, create a PropagateDownload job and compute the local hash. If the hashes are identical, don't download the file and just update metadata. * Avoid exposing the existence of checksumTypeId beyond the database layer. This makes handling checksums easier in general because they can usually be treated as a single blob. This change was prompted by the difficulty of producing file_stat_t entries uniformly from PROPFINDs and the database.
2017-06-14 13:14:46 +03:00
void testFakeConflict()
{
FakeFolder fakeFolder{ FileInfo::A12_B12_C12_S12() };
int nGET = 0;
fakeFolder.setServerOverride([&](QNetworkAccessManager::Operation op, const QNetworkRequest &) {
if (op == QNetworkAccessManager::GetOperation)
++nGET;
return nullptr;
});
// For directly editing the remote checksum
FileInfo &remoteInfo = dynamic_cast<FileInfo &>(fakeFolder.remoteModifier());
// Base mtime with no ms content (filesystem is seconds only)
Use QDateTime::currentDateTimeUtc instead of the non utc version QDateTime::currentDateTime is terribly slow.
2017-09-25 12:49:11 +03:00
auto mtime = QDateTime::currentDateTimeUtc().addDays(-4);
Compare the hash of files with identical mtime/size #5589 * For conflicts where mtime and size are identical: a) If there's no remote checksum, skip (unchanged) b) If there's a remote checksum that's a useful hash, create a PropagateDownload job and compute the local hash. If the hashes are identical, don't download the file and just update metadata. * Avoid exposing the existence of checksumTypeId beyond the database layer. This makes handling checksums easier in general because they can usually be treated as a single blob. This change was prompted by the difficulty of producing file_stat_t entries uniformly from PROPFINDs and the database.
2017-06-14 13:14:46 +03:00
mtime.setMSecsSinceEpoch(mtime.toMSecsSinceEpoch() / 1000 * 1000);
// Conflict: Same content, mtime, but no server checksum
// -> ignored in reconcile
fakeFolder.localModifier().setContents("A/a1", 'C');
fakeFolder.localModifier().setModTime("A/a1", mtime);
fakeFolder.remoteModifier().setContents("A/a1", 'C');
fakeFolder.remoteModifier().setModTime("A/a1", mtime);
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nGET, 0);
// Conflict: Same content, mtime, but weak server checksum
// -> ignored in reconcile
mtime = mtime.addDays(1);
fakeFolder.localModifier().setContents("A/a1", 'D');
fakeFolder.localModifier().setModTime("A/a1", mtime);
fakeFolder.remoteModifier().setContents("A/a1", 'D');
fakeFolder.remoteModifier().setModTime("A/a1", mtime);
remoteInfo.find("A/a1")->checksums = "Adler32:bad";
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nGET, 0);
// Conflict: Same content, mtime, but server checksum differs
// -> downloaded
mtime = mtime.addDays(1);
fakeFolder.localModifier().setContents("A/a1", 'W');
fakeFolder.localModifier().setModTime("A/a1", mtime);
fakeFolder.remoteModifier().setContents("A/a1", 'W');
fakeFolder.remoteModifier().setModTime("A/a1", mtime);
remoteInfo.find("A/a1")->checksums = "SHA1:bad";
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nGET, 1);
// Conflict: Same content, mtime, matching checksums
// -> PropagateDownload, but it skips the download
mtime = mtime.addDays(1);
fakeFolder.localModifier().setContents("A/a1", 'C');
fakeFolder.localModifier().setModTime("A/a1", mtime);
fakeFolder.remoteModifier().setContents("A/a1", 'C');
fakeFolder.remoteModifier().setModTime("A/a1", mtime);
remoteInfo.find("A/a1")->checksums = "SHA1:56900fb1d337cf7237ff766276b9c1e8ce507427";
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nGET, 1);
// Extra sync reads from db, no difference
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nGET, 1);
}
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
/**
* Checks whether SyncFileItems have the expected properties before start
* of propagation.
*/
void testSyncFileItemProperties()
{
Use QDateTime::currentDateTimeUtc instead of the non utc version QDateTime::currentDateTime is terribly slow.
2017-09-25 12:49:11 +03:00
auto initialMtime = QDateTime::currentDateTimeUtc().addDays(-7);
auto changedMtime = QDateTime::currentDateTimeUtc().addDays(-4);
auto changedMtime2 = QDateTime::currentDateTimeUtc().addDays(-3);
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
// Base mtime with no ms content (filesystem is seconds only)
initialMtime.setMSecsSinceEpoch(initialMtime.toMSecsSinceEpoch() / 1000 * 1000);
changedMtime.setMSecsSinceEpoch(changedMtime.toMSecsSinceEpoch() / 1000 * 1000);
changedMtime2.setMSecsSinceEpoch(changedMtime2.toMSecsSinceEpoch() / 1000 * 1000);
SyncEngineTest: Fix test reliability There was a rounding issue in the mtimes which sometimes resulted in an off-by-one error. Caused by storing a full QDateTime in the FileInfo but the mtime saved to the disk being truncated to seconds.
2017-07-04 10:57:22 +03:00
// Ensure the initial mtimes are as expected
auto initialFileInfo = FileInfo::A12_B12_C12_S12();
initialFileInfo.setModTime("A/a1", initialMtime);
initialFileInfo.setModTime("B/b1", initialMtime);
initialFileInfo.setModTime("C/c1", initialMtime);
FakeFolder fakeFolder{ initialFileInfo };
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
// upload a
fakeFolder.localModifier().appendByte("A/a1");
fakeFolder.localModifier().setModTime("A/a1", changedMtime);
// download b
fakeFolder.remoteModifier().appendByte("B/b1");
fakeFolder.remoteModifier().setModTime("B/b1", changedMtime);
// conflict c
fakeFolder.localModifier().appendByte("C/c1");
fakeFolder.localModifier().appendByte("C/c1");
fakeFolder.localModifier().setModTime("C/c1", changedMtime);
fakeFolder.remoteModifier().appendByte("C/c1");
fakeFolder.remoteModifier().setModTime("C/c1", changedMtime2);
connect(&fakeFolder.syncEngine(), &SyncEngine::aboutToPropagate, [&](SyncFileItemVector &items) {
SyncFileItemPtr a1, b1, c1;
for (auto &item : items) {
if (item->_file == "A/a1")
a1 = item;
if (item->_file == "B/b1")
b1 = item;
if (item->_file == "C/c1")
c1 = item;
}
// a1: should have local size and modtime
QVERIFY(a1);
QCOMPARE(a1->_instruction, CSYNC_INSTRUCTION_SYNC);
QCOMPARE(a1->_direction, SyncFileItem::Up);
SyncEngineTest: _size now correct for uploads #5855
2017-07-12 11:09:04 +03:00
QCOMPARE(a1->_size, quint64(5));
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
QCOMPARE(Utility::qDateTimeFromTime_t(a1->_modtime), changedMtime);
Remove SyncFileItem::log This remove the remaining "other" fields of the sync log to save a bit of memory. other_etag and other_fileId don't give much information to the users and other_instruction will always be INST_NONE anyway. other_modtime and other_size are kept since they are sometimes used. They were renamed to have a bit more meaningful name. SyncEngine::checkPermissions will now fetch its information from the csync trees since they are now preserved until right after this point. Fixes #3213
2017-08-24 18:07:22 +03:00
QCOMPARE(a1->_previousSize, quint64(4));
QCOMPARE(Utility::qDateTimeFromTime_t(a1->_previousModtime), initialMtime);
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
// b2: should have remote size and modtime
QVERIFY(b1);
QCOMPARE(b1->_instruction, CSYNC_INSTRUCTION_SYNC);
QCOMPARE(b1->_direction, SyncFileItem::Down);
QCOMPARE(b1->_size, quint64(17));
QCOMPARE(Utility::qDateTimeFromTime_t(b1->_modtime), changedMtime);
Remove SyncFileItem::log This remove the remaining "other" fields of the sync log to save a bit of memory. other_etag and other_fileId don't give much information to the users and other_instruction will always be INST_NONE anyway. other_modtime and other_size are kept since they are sometimes used. They were renamed to have a bit more meaningful name. SyncEngine::checkPermissions will now fetch its information from the csync trees since they are now preserved until right after this point. Fixes #3213
2017-08-24 18:07:22 +03:00
QCOMPARE(b1->_previousSize, quint64(16));
QCOMPARE(Utility::qDateTimeFromTime_t(b1->_previousModtime), initialMtime);
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
// c1: conflicts are downloads, so remote size and modtime
QVERIFY(c1);
QCOMPARE(c1->_instruction, CSYNC_INSTRUCTION_CONFLICT);
QCOMPARE(c1->_direction, SyncFileItem::None);
QCOMPARE(c1->_size, quint64(25));
QCOMPARE(Utility::qDateTimeFromTime_t(c1->_modtime), changedMtime2);
Remove SyncFileItem::log This remove the remaining "other" fields of the sync log to save a bit of memory. other_etag and other_fileId don't give much information to the users and other_instruction will always be INST_NONE anyway. other_modtime and other_size are kept since they are sometimes used. They were renamed to have a bit more meaningful name. SyncEngine::checkPermissions will now fetch its information from the csync trees since they are now preserved until right after this point. Fixes #3213
2017-08-24 18:07:22 +03:00
QCOMPARE(c1->_previousSize, quint64(26));
QCOMPARE(Utility::qDateTimeFromTime_t(c1->_previousModtime), changedMtime);
SyncEngine: Add unittest for SyncFileItem properties #5855 Checks instruction, direction, size, modtime for three common cases.
2017-06-28 12:15:22 +03:00
});
QVERIFY(fakeFolder.syncOnce());
}
PropagateUpload: Model of remote quota, avoid some uploads #5537 When we see a 507 error, assume that quota is < uploaded size.
2017-07-12 10:58:15 +03:00
/**
* Checks whether subsequent large uploads are skipped after a 507 error
*/
void testInsufficientRemoteStorage()
{
FakeFolder fakeFolder{ FileInfo::A12_B12_C12_S12() };
// Disable parallel uploads
SyncOptions syncOptions;
syncOptions._parallelNetworkJobs = false;
fakeFolder.syncEngine().setSyncOptions(syncOptions);
// Produce an error based on upload size
int remoteQuota = 1000;
int n507 = 0, nPUT = 0;
auto parent = new QObject;
fakeFolder.setServerOverride([&](QNetworkAccessManager::Operation op, const QNetworkRequest &request) -> QNetworkReply * {
if (op == QNetworkAccessManager::PutOperation) {
nPUT++;
if (request.rawHeader("OC-Total-Length").toInt() > remoteQuota) {
n507++;
return new FakeErrorReply(op, request, parent, 507);
}
}
return nullptr;
});
fakeFolder.localModifier().insert("A/big", 800);
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nPUT, 1);
QCOMPARE(n507, 0);
nPUT = 0;
fakeFolder.localModifier().insert("A/big1", 500); // ok
fakeFolder.localModifier().insert("A/big2", 1200); // 507 (quota guess now 1199)
fakeFolder.localModifier().insert("A/big3", 1200); // skipped
fakeFolder.localModifier().insert("A/big4", 1500); // skipped
fakeFolder.localModifier().insert("A/big5", 1100); // 507 (quota guess now 1099)
fakeFolder.localModifier().insert("A/big6", 900); // ok (quota guess now 199)
fakeFolder.localModifier().insert("A/big7", 200); // skipped
fakeFolder.localModifier().insert("A/big8", 199); // ok (quota guess now 0)
fakeFolder.localModifier().insert("B/big8", 1150); // 507
QVERIFY(!fakeFolder.syncOnce());
QCOMPARE(nPUT, 6);
QCOMPARE(n507, 3);
}
Rename detection: File size must be equal Comparison of file sizes for potential conflicts was added in 0eb9401c624f20a128b46f8eb1fa5a984f9ef61e, but did not extend to checking the file size in case of potential local moves. This commit adds this check and adds tests for various move+change scenarios.
2017-10-05 12:39:35 +03:00
void testLocalMove()
{
FakeFolder fakeFolder{ FileInfo::A12_B12_C12_S12() };
int nPUT = 0;
int nDELETE = 0;
fakeFolder.setServerOverride([&](QNetworkAccessManager::Operation op, const QNetworkRequest &) {
if (op == QNetworkAccessManager::PutOperation)
++nPUT;
if (op == QNetworkAccessManager::DeleteOperation)
++nDELETE;
return nullptr;
});
// For directly editing the remote checksum
FileInfo &remoteInfo = fakeFolder.remoteModifier();
// Simple move causing a remote rename
fakeFolder.localModifier().rename("A/a1", "A/a1m");
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
QCOMPARE(nPUT, 0);
// Move-and-change, causing a upload and delete
fakeFolder.localModifier().rename("A/a2", "A/a2m");
fakeFolder.localModifier().appendByte("A/a2m");
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
QCOMPARE(nPUT, 1);
QCOMPARE(nDELETE, 1);
// Move-and-change, mtime+content only
fakeFolder.localModifier().rename("B/b1", "B/b1m");
fakeFolder.localModifier().setContents("B/b1m", 'C');
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
QCOMPARE(nPUT, 2);
QCOMPARE(nDELETE, 2);
// Move-and-change, size+content only
auto mtime = fakeFolder.remoteModifier().find("B/b2")->lastModified;
fakeFolder.localModifier().rename("B/b2", "B/b2m");
fakeFolder.localModifier().appendByte("B/b2m");
fakeFolder.localModifier().setModTime("B/b2m", mtime);
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
QCOMPARE(nPUT, 3);
QCOMPARE(nDELETE, 3);
// Move-and-change, content only -- c1 has no checksum, so we fail to detect this!
mtime = fakeFolder.remoteModifier().find("C/c1")->lastModified;
fakeFolder.localModifier().rename("C/c1", "C/c1m");
fakeFolder.localModifier().setContents("C/c1m", 'C');
fakeFolder.localModifier().setModTime("C/c1m", mtime);
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nPUT, 3);
QCOMPARE(nDELETE, 3);
QVERIFY(!(fakeFolder.currentLocalState() == remoteInfo));
// cleanup, and upload a file that will have a checksum in the db
fakeFolder.localModifier().remove("C/c1m");
fakeFolder.localModifier().insert("C/c3");
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
QCOMPARE(nPUT, 4);
QCOMPARE(nDELETE, 4);
// Move-and-change, content only, this time while having a checksum
mtime = fakeFolder.remoteModifier().find("C/c3")->lastModified;
fakeFolder.localModifier().rename("C/c3", "C/c3m");
fakeFolder.localModifier().setContents("C/c3m", 'C');
fakeFolder.localModifier().setModTime("C/c3m", mtime);
QVERIFY(fakeFolder.syncOnce());
QCOMPARE(nPUT, 5);
QCOMPARE(nDELETE, 5);
QCOMPARE(fakeFolder.currentLocalState(), remoteInfo);
}
Add TestSyncEngine and TestSyncFileStatusTracker auto tests To be able to test the SyncEngine efficiently, a set of server mocking classes have been implemented on top of QNetworkAccessManager. The local disk side hasn't been mocked since this would require adding a large abstraction layer in csync. The SyncEngine is instead pointed to a different temporary dir in each test and we test by interacting with files in this directory instead. The FakeFolder object wraps the SyncEngine with those abstractions and allow controlling the local files, and the fake remote state through the FileModifier interface, using a FileInfo tree structure for the remote-side implementation as well as feeding and comparing the states on both side in tests. Tests run fast and require no setup to be run, but each server feature that we want to test on the client side needs to be implemented in this fake objects library. For example, the OC-FileId header isn't set as of this commit, and we can't test the file move logic properly without implementing it first. The TestSyncFileStatusTracker tests already contain a few QEXPECT_FAIL for what I esteem being issues that need to be fixed in order to catch up on our test coverage without making this patch too huge.
2016-08-04 15:59:46 +03:00
};
QTEST_GUILESS_MAIN(TestSyncEngine)
#include "testsyncengine.moc"
Reference in a new issue Copy permalink