Commit graph

2693 commits

Author SHA1 Message Date
Tone
87fa6ea71e
[HeiseBridge.php] Prevent Youtube videos from being filtered out (#4125) 2024-06-10 19:40:07 +02:00
Tim-Florian Feulner
36706a3dec
Fix NACSouthGermanyMediaLibraryBridge due to website changes (#4121) 2024-06-03 00:55:39 +02:00
tillcash
cfd406861e
[HarvardHealthBlogBridge] Update (#4117)
Make article image optional as all images are representative
2024-05-30 16:08:08 +02:00
tillcash
bd90109c70
[HarvardHealthBlogBridge] New (#4116) 2024-05-29 21:16:10 +02:00
tillcash
5a68ee0c87
[HinduTamilBridge] New (#4115) 2024-05-26 17:21:14 +02:00
Albert Kiskorov
dc199ebf5c
Fix: Ensure $time is set from innertext when datetime attribute is not found (#4111)
This commit addresses a bug where the $time variable is not set from the innertext of the $time_element when the datetime attribute is not found. The previous implementation only checked if $time was null or an empty string, which did not cover all cases where the datetime attribute might be missing. By using the empty() function, we ensure that $time is correctly set from the innertext when the datetime attribute is not present.
2024-05-19 14:37:59 +02:00
Mynacol
75f35391fa
[HeiseBridge] Add missing <ol> elements (#4110)
The following article has <ol> elements that were missing.
Adding them to have the full content.

https://heise.de/-9714438
2024-05-18 16:51:00 +02:00
Mynacol
7bde7a56f9 [ZeitBridge] Fix linting 2024-05-18 16:35:24 +02:00
Mynacol
4d12aa2a9e [ZeitBridge] Remove annoyances, add content
Remove navigational elements, podcast images.
Add many more header images, article content in <ul> (and for ggod
measure in <ol>) and quotes with their content and not only their
author.

Extreme example:
https://www.zeit.de/campus/2024-05/protest-palaestina-universitaet-europa-uebersicht
2024-05-18 16:35:24 +02:00
Mynacol
a7ed3d56f9 [ZeitBridge] Prettify author field
By removing HTML tags (plaintext) and trimming it.
2024-05-18 16:35:24 +02:00
July
b785a4b64e
ArsTechnicaBridge: restore categories lost by FeedExpander (#4030) 2024-05-17 21:29:17 +02:00
July
6e2aeda61d
[GameBananaBridge] Include update contents in feed (#4103)
* [GameBananaBridge] Include update contents in feed

* [GameBananaBridge] Fix dynamic title property
2024-05-12 21:46:07 +02:00
July
4949900863
[ScribbleHubBridge] Handle 429 errors and use consistent GUID (#4104) 2024-05-12 21:45:14 +02:00
Alex Balgavy
776ee233bd
[NOSBridge] fix bridge (#4102)
CSS selectors were no longer valid.
2024-05-12 20:30:23 +02:00
Facundo Tuesca
1c3024fca7
[MangaReaderBridge] Change feed title to manga name (#4092) 2024-05-08 00:25:45 +02:00
Patrick
d11b7f7754
Change URI for St. Johannes Blick (#4099)
Co-authored-by: Patrick <jummo@mailbox.org>
2024-05-05 23:30:38 +02:00
Eugene Molotov
f480209825
[YoutubeBridge] Fix empty result in search feed (#4098) 2024-05-05 23:30:23 +02:00
Thomas
d15960f955
[YouTubeCommunityTabBridge] Multi-image attachment support (#4091)
Adds support for multi-image attachments.
Also changes individual if-statments in "getAttachments" to if/elseif
as each post can apparently only have one attachment anyway.
2024-05-02 19:45:04 +02:00
Korytov Pavel
f3ca567159
[TldrTechBridge] Fix and improve bridge (#4090) 2024-04-27 10:35:59 +02:00
Thomas
d31f20758c
[YouTubeCommunityTabBridge] Improve building of content & title (#4089)
* [YouTubeCommunityTabBridge] Improve building of content & title

Fixes truncated link hrefs in content and adds some general
improvements regarding the building of item content and item title

* [YouTubeCommunityTabBridge] Fix PHP deprecation warnings

Fixes the following deprecation warnings:

substr(): Passing null to parameter #1 ($string) of type string is
deprecated
2024-04-26 18:47:06 +02:00
Tone
154b8b9cdb
Create TarnkappeBridge.php (#4085)
* Create TarnkappeBridge.php

* Update TarnkappeBridge.php
2024-04-19 19:08:58 +02:00
Mynacol
1f71d76ac1 [HeiseBridge] Remove additional ad banners
For example
https://www.heise.de/meinung/Kommentar-Microsofts-Sicherheitspraxis-wird-zur-Gefahr-und-das-BSI-schweigt-9686629.html
has two inline banners for a heise offering, not directly related to the
article. Removing all "inline" figures, which seems to catch all inline
unwanted elements, while avoiding removing useful figures/images.
2024-04-18 13:39:37 +02:00
sysadminstory
8c3e973b9f
[PepperBridgeAbstract] Fix the "no result" detection (#4082)
The "no result" test did not work, it is fixed now.
2024-04-18 01:43:53 +02:00
llamasblade
97f5dafbc5
[HytaleBridge] Fix bridge not pulling all blog posts (#4079) 2024-04-16 17:58:05 +02:00
llamasblade
957a820931
[YandexZenBridge] Fix broken bridge for some channels (#4078)
Fixes #4071.

Major changes:
- the bridge's URI changed from zen.yandex.com to dzen.ru, as the former
  redirects to the latter (perhaps the bridge's name should be changed
  as well);
- the channel's URL is now required instead of the channel's username;
- two kinds of URLs are supported, one for channels with usernames and
  one for channels with IDs in their URL;
- the channel's real name, as shown in the webpage, is now used as the
  feed title.
2024-04-14 19:14:52 +02:00
Miika Launiainen
b4d397ff70
[YorushikaBridge] Fix getting date (#4077)
* Remove unnecessary variable

* Fix getting date
2024-04-14 19:13:31 +02:00
Korytov Pavel
428c6c3c66
[ScientificAmericanBridge] Update bridge (#4074)
* [ScientificAmericanBridge] Update bridge

* [ScientificAmericanBridge] Fix lint
2024-04-12 01:57:55 +02:00
Miika Launiainen
58c254ad3b
[YorushikaBridge] Add language selection parameter (#4073)
* Add language selection parameter

* Fix typo

* Fix lint errors
2024-04-11 17:18:37 +02:00
Dag
a73b66f4d6
fix(ScientificAmericanBridge) (#4070) 2024-04-10 18:32:48 +02:00
sysadminstory
815dc180cc
[PicukiBridge] Fix image URL (#4068)
Image URL does not need to be faked anymore, as the content/type is now valid.
2024-04-10 17:30:56 +02:00
July
7d6881732d
[ScribbleHubBridge] Add list page feed creation (#4012)
* [ScribbleHubBridge] Add list page feed creation

* [ScribbleHubBridge] Add list title handling

* [ScribbleHubBridge] Don't include timestamp in List GUIDs

* [ScribbleHubBridge] Fix usage of dynamic property
2024-04-07 23:02:36 +02:00
Dag
4602f4f475
tweaks (#4065) 2024-04-06 18:07:45 +02:00
Mynacol
b3ac1d176c
[FDroidRepoBridge] Simplify json retrieval (#4063)
* [FDroidRepoBridge] Simplify json retrieval

I looked into avoiding the writing-to-file and then reading-from-file altogether. Using a special file path that leaves the data in memory probably wouldn't work. But I'm unsure why we use the `index-v1.jar` file altogether.
The main F-Droid repo [lists](https://f-droid.org/en/docs/All_our_APIs/#the-repo-index) not only `index-v1.jar` (which only makes sense if we were to use the contained signature, which we don't), but also `index-v1.json` and `index-v2.json`. These json files can be fetched with `getContents`, optionally cached, and directly fed into `Json::decode` without using a temporary file. The HTTP transfer encoding can compress the file to a similar degree the jar (=zip) can. That's exactly what this commit uses.

Now the question is whether all the F-Droid repositories out there have this file. I went through the whole [list of known repositories](https://forum.f-droid.org/t/known-repositories/721) and only one repo misses the `index-v1.json` file: [Bromite](https://fdroid.bromite.org/fdroid/repo/index-v1.json). Under these circumstances we can depend on the availability of the `index-v1.json` file.

Closes #4062

* [FDroidRepoBridge] Cleanup not requiring Zip

With the last commit 1152386678, the zip
extension is not required anymore. Don't fail if it's not available.
2024-04-05 17:39:38 +02:00
Mynacol
d5aa3aef69 [FDroidRepoBridge] Fix example repo
The ttrss example/placeholder repo is offline, which fails CI jobs.
Replace it with a healthy repo and package to get working CI tests and comparisons.
2024-04-05 11:39:43 +02:00
Dag
001dd47439
fix: small tweaks (#4057) 2024-04-04 19:12:04 +02:00
Dag
3cba984d22
fix(FDroidRepoBridge): unlink when json file is absent from archive (#4056) 2024-04-04 17:43:07 +02:00
sysadminstory
82606a479a
[PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling (#4053)
* [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling

Search URL has been updated according to the website.

If a search doesn't return any results, the HTML won't contain any
specific text now : the HTML structure is slightly different, so the
bridge has been updated.

The unnneded 'no-results' text is now removed from the specific bridges.

The board thread title has been removed from the content, so now we use
the page <title> element.

In case a board message is empty, there was an exception during the
filtering of message without URL.

* [PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix search URL, No results handling fixed, Thread title and Message URL handling

Coding policy fixes
2024-04-04 04:08:29 +02:00
Tone
f736da6fae
[GolemBridge] fix for internal videos (#4051)
* [GolemBridge] fix for internal videos

with this internal golem-videos can be played directly from feed

* Update GolemBridge.php
2024-04-03 16:23:52 +02:00
Niehztog
fb66775ece
[XPathAbstract] Refactor xpath abstract (#4047)
* refactor XPathAbstract, keep all functionality intact

* fix linter errors

* further simplify code

* set default value for raw item content to true, avoiding escaping of html tags in feed item contents by default
2024-04-02 23:14:25 +02:00
Dawid Wróbel
8f962383c2
[eBayBridge] fix Belgian eBay URL handling (#4050)
Fixes #3918
2024-04-02 01:01:23 +02:00
Dawid Wróbel
bb979e9e08
[AllegroBridge] fix logical condition on parameters (#4049) 2024-04-02 00:06:15 +02:00
Dawid Wróbel
a12bab9eed
[AllegroBridge] ask for a complete cookie string, mere wcdx works no more (#4048) 2024-04-01 23:44:45 +02:00
Miika Launiainen
b4659786cb
[GenshinImpactBridge] Small fixes (#4046)
* Switch json_decode to Json::decode

* Change regex delimeter from / to #

* Save item enclosures as list
2024-04-01 21:16:32 +02:00
July
7001fbaf49
[AO3Bridge] Fix bad heading selector (#4045) 2024-03-31 22:41:58 +02:00
Dag
d5d470cbc2
fix(dribble) (#4044) 2024-03-31 22:10:59 +02:00
Dag
182567e434
fix(bridges/DavesTrailerPageBridge): remove (#4043) 2024-03-31 21:52:53 +02:00
Dag
9682f74fc5
fix(cnet): author typo (#4042) 2024-03-31 21:37:51 +02:00
Dag
17a3b4c9d8
Fix 198 (#4041)
* fix(twitch): log instead of exception

* typo
2024-03-31 21:32:27 +02:00
Dag
73289324bd
feat: add vendor http header to cached responses (#4040) 2024-03-31 21:02:55 +02:00
Dag
8ca1b90840
fix(NationalGeographicBridge) (#4039) 2024-03-31 20:07:14 +02:00
Niehztog
1c3c85d8ff
[XPathBridge] Allow multiple categories (#4038)
* [XPathAbstract] allow multiple categories

* fix feed icons in two bridges

* fix warning

* fix linter errors
2024-03-31 18:46:07 +02:00
Miika Launiainen
d23fd2522c
[GenshinImpactBridge] Fix bridge to use new API (#4011)
* [GenshinImpactBridge] Fix bridge to use new API

* Add category parameters back to not break existing feeds

* Fix lint error

* Remove whitespace
2024-03-31 03:46:23 +02:00
Dag
545dc969d3
refactor (#4037) 2024-03-31 03:38:42 +02:00
Tone
e0be366258
Update AnisearchBridge.php (#4025)
* Update AnisearchBridge.php

added youtube trailer

* made trailers optional and reduced scraping to 5 articles if selected

* Update AnisearchBridge.php
2024-03-29 15:37:43 +01:00
sysadminstory
be445759b6
[PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Move as much as possible to JSON (#4032)
As the website use more and more JSON, and JSON is a machine readable
format, I migrated as much as possible to the JSON.

This simplifies the Abstract class a lot, and the Bridge classes need
less language specifi strings.
2024-03-28 19:44:27 +01:00
July
db984d8a8b
AO3Bridge: move tags to categories and remove duplicate fic summary (#4031)
* AO3Bridge: move tags to categories and remove duplicate fic summary

* [AO3Bridge] Fix tag html entity encoding
2024-03-28 19:43:17 +01:00
Tone
e251e358ff
[HeiseBridge] fix for embedded youtube-videos (#4034)
* [HeiseBridge] fix for embbedded youtube-videos

with this the embedded youtube videos will work in the feed

* Update HeiseBridge.php

* Update HeiseBridge.php
2024-03-28 19:42:41 +01:00
Tone
0c2099a852
[GolemBridge] fixed embedded youtube videos (#4033)
* [GolemBridge] fixed embedded youtube videos

embedded youtube-videos can be played directly from feed now

* Update GolemBridge.php

* Update GolemBridge.php

* Update GolemBridge.php

* Update GolemBridge.php
2024-03-28 19:41:56 +01:00
Tone
fee5e269d0
Update CaschyBridge.php (#4027)
without removing the video-container-div the embedded youtube videos work again
2024-03-24 16:38:51 +01:00
Tone
2aace6c898
Added Bridge for Anisearch.de (#4023)
* Create AnisearchBridge.php

* Update AnisearchBridge.php

* Update AnisearchBridge.php
2024-03-22 21:01:16 +01:00
sysadminstory
3ed193eee2
[IdealoBridge] Update Bridge Meta data & (#4022)
The bridge meta data has been updated to reflect that the bridge works
for other international version of Idealo.

The Price trend is displayed on every price in the the Feed element
content. The same function is now used to show the price trend in the
Feed element title, to remove some duplicate code..
2024-03-22 09:44:42 +01:00
Patrick
58e2b56d40
Adjustment to new website layout (#4020) 2024-03-17 19:03:09 +01:00
Tone
a61524bf77
Update RedditBridge.php (#4019)
prevent error htmlspecialchars_decode(): Passing null to parameter #1
2024-03-17 19:02:51 +01:00
Tim-Florian Feulner
36147a082d
Fix NACSouthGermanyMediaLibraryBridge for new website layout (#4014) 2024-03-15 19:20:04 +01:00
sysadminstory
e6cb5fdc89
[IdealoBridge] Fix Feed items & Feed title customisation (#4013)
- Feed items with new price tracking had "Max Price Used" instead of
  "Max Price New"
- Feed Title is now customised with the product name and the Price
  limits
- Fixed logic for saving prices in cache
- remove undefined variable notices
2024-03-13 23:47:46 +01:00
Dag
4bad1c140a
fix(reddit): url encoding (#4010) 2024-03-12 23:59:10 +01:00
tillcash
ecf61f6fa7
[DailythanthiBridge] New Bridge (#4006) 2024-03-11 20:14:10 +01:00
Mynacol
254efc2812 [ZeitBridge] Remove doubled text
The first two paragraphs were repeated at the end of articles. The first
CSS selector filters those out (example 1).
The second CSS selector removes a "Zum Anschauen benötigen wir Ihre Zustimmung"
line from a poll widget. We can't load the widget successfully,
therefore we should remove all embeds that seem to use javascript
(example 2).

1: https://www.zeit.de/campus/2024-03/bundesregierung-wissenschaft-arbeitsvertrag-regeln
2: https://www.zeit.de/campus/2024-03/ausbildung-abgebrochen-gruende-azubi-aufruf
2024-03-10 22:27:32 +01:00
Jonathan Kay
84b93e0f8f
[ComicsKingdomBridge] Fix/Rewrite of ComicsKingdom Bridge (#4003)
* Rewrite ComicsKingdom Bridge

Rewrite of bridge as the existing one no longer works:
- Now uses REST API
- Added optional limit to get desired number of comics
- Author now reflects the comic creators name
- Feed name and comic titles now pulled from site
- Added myself as the maintainer as I've been the one maintaining, and the existing code no longer is used

* Change API to URI to pass test

* Remove whitespace, add curly braces and switch to single quotes
2024-03-10 15:18:50 +01:00
tillcash
79699131e8
[MaalaimalarBridge] New Bridge (#4001) 2024-03-08 12:46:32 +01:00
July
f7c1b71939
NyaaTorrentsBridge: add torrent to enclosures and generate better feed name (#3996)
* NyaaTorrentsBridge: add torrent to enclosures and generate better feed name

* NyaaTorrentsBridge: fix accidental () in bridge name
2024-03-06 19:40:59 +01:00
July
7a7f8d5050
AnnasArchiveBridge: correctly handling partial matches and file links (#3997) 2024-03-06 01:28:24 +01:00
D5k H3h
683c968d64
[Rooster Teeth] Add Camp Camp channel (#3992) 2024-03-01 20:24:14 +01:00
hleskien
932f20d434
fixed date with time in LuftfahrtBundesAmtBridge (#3987) 2024-02-18 19:19:33 +01:00
Korytov Pavel
e65155f440
[OpenCVEBridge] Add bridge (#3978)
* [OpenCVEBridge] Add bridge

* [OpenCVEBridge] Fix tests

* [OpenCVEBridge] Fix description of the filter parameter
2024-02-16 22:24:13 +01:00
July
7813f4564e
AO3Bridge: add options to fetch chapter contents and list titles (#3981)
* AO3Bridge: add options to fetch chapter contents and titles for list feeds

and add downloads for each fic to enclosures

* AO3Bridge: fix list default value

* AO3Bridge: fix erroneous dynamic property usage

* AO3Bridge: fix unit test failure for getURI
2024-02-16 04:14:17 +01:00
sysadminstory
4d15ffd2cf
[PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] (#3982)
Exclude thread results

Some categories showed some thread in the middle of the deals : now only
the deals are handled

Updated the "no results" text to follow the sites changes
2024-02-16 03:58:15 +01:00
Dag
598ee5b51e
fix(pinterest): set enclosure so it emits mrss media:content prop (#3980) 2024-02-14 16:02:54 +01:00
Eugene Molotov
257799be8e
[Vk2Bridge] Alternative bridge for VK (#3878) 2024-02-10 15:59:39 +01:00
hleskien
8e8028b786
Adopt WebDriverAbstract as a solution for active (JavaScript) websites (#3971)
* first working version

---------

Co-authored-by: Dag <me@dvikan.no>
2024-02-10 04:42:22 +01:00
tillcash
46ac77590e
[KilledbyMicrosoftBridge] Update: Adjusted content format for consistency (#3968) 2024-02-09 09:39:03 +01:00
Dag
6f731b20a9
fix(DarkReading): official rss endpoint changed (#3967) 2024-02-09 08:03:04 +01:00
Dag
8a6798a227
fix: escape token for html context (#3966) 2024-02-09 07:27:16 +01:00
Tone
ae2eb2f1d1
feat(Reddit): add parameter for web UI frontend 2024-02-08 20:05:24 +01:00
Korytov Pavel
cfef482366
[EconomistBridge] Handle 404s in feed gracefully (#3965) 2024-02-08 15:36:03 +01:00
Tone
75a0a779c0
Update HeiseBridge.php (#3963)
fix for broken article categories
2024-02-08 15:35:24 +01:00
tillcash
6bb04d48ed
[KilledbyMicrosoftBridge] New Bridge (#3961) 2024-02-07 19:33:25 +01:00
Dag
6878eb26aa
fix: changed dom (#3958) 2024-02-06 19:32:05 +01:00
sysadminstory
64f95b4990
[PepperBridgeAbstract,DealabsBridge,HotUKDealsBridge,MydealsBridge] Fix missing price, discount and ships from information (#3956)
- DealabsBridge
- HotUKDealsBridge
- MydealsBridge
Add the currency in the i8n data of the bridges

- PepperBridgeAbstract
The Price, discount data ans Ships from information are in the HTML
content anymore, so switched to the js-vue2 attributes
2024-02-06 02:23:12 +01:00
Scott Colby
66a6847fd0
Two fixes to DeutscheWelle (#3954)
* [DeutscheWelleBridge] Small URL fix.

Reset the $item's uri value after removing the tracking query string.

* [DeutscheWelleBridge] Fix "hero" images.

The main "hero" image for each article has src="" and relies on the
srcset attribute for the browser to pick the best image based on the
actual displayed size.

The call to `defaultLinkTo()` replaces the empty src with the article's
link, which, not being an image, breaks the image.

This change resets the src's of any such images back to "".
2024-02-06 02:21:30 +01:00
sysadminstory
7931f37a83
[PepperBridgeAbstract] Fix deal image scraping (#3953)
Deal Image was moved to a vuejs element, the deal image scraping was
fixed.
2024-02-05 23:30:18 +01:00
Tostiman
d175bab58e
Fix car throttle bridge (#3925) 2024-02-04 18:28:12 +01:00
Clemens Neubauer
81be72ea04
Update BMDSystemhausBlogBridge.php (#3943) 2024-02-02 15:33:59 +01:00
Dag
d01c462ad5
fix(FeedExpander): if parse fails, include offending url in exception message (#3938)
Also some refactors
2024-01-29 21:51:34 +01:00
Dag
b2c8475b2c
fix(gates): json decoding (#3937) 2024-01-29 21:51:23 +01:00
tillcash
a15e578158
[FirefoxReleaseNotesBridge] Add New Bridge (#3930)
* [FirefoxReleaseNotesBridge] Add New Bridge

I'm uncertain about the reasons for the failed checks.

* Update FirefoxReleaseNotesBridge.php
2024-01-27 13:44:58 +01:00
Dag
d08d13f2c8
refactor: introduce http Request object (#3926) 2024-01-25 16:06:24 +01:00
Dag
9574c17ddc
refactor/fix (#3924) 2024-01-25 13:03:00 +01:00
hleskien
1262cc982c
added new bridges (#3920)
* added new bridges

* lint

---------

Co-authored-by: Dag <me@dvikan.no>
2024-01-24 19:12:38 +01:00
Dag
487c692e68
fix: a few deprecation notices on php 8.2 (#3917)
* fix: a few deprecation notices on php 8.2

* tweak
2024-01-23 23:02:06 +01:00