Commit graph

2475 commits

Author SHA1 Message Date
LogMANOriginal
cdc1d9c9ba
action: Add action to check bridge connectivity (#1147)
* action: Add action to check bridge connectivity

It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.

In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.

If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.

* [Connectivity] Use Bootstrap container to properly display contents

* [Connectivity] Limit connectivity checks to debug mode

Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).

* [Connectivity] Split implemenation into multiple files

* [Connectivity] Make web page responsive to user input

* [Connectivity] Make status message sticky

* [Connectivity] Add icon to the status message

* [contents] Add the ability for getContents to return header information

* [Connectivity] Add header information to the reply Json data

* [Connectivity] Add new status (blue) for redirected sites

Also adds titles to status icons (Successful, Redirected, Inactive, Failed)

* [Connectivity] Fix show doesn't work for inactive bridges

* [Connectivity] Fix typo

* [Connectivity] Catch errors in promise chains

* [Connectivity] Allow search by status and update dynamically

* [Connectivity] Add a progress bar

* [Connectivity] Use bridge factory

* [Connectivity] Import Bootstrap v4.3.1 CSS
2019-10-31 22:02:38 +01:00
LogMANOriginal
6bc83310b9
core: Add info button for input fields with title (#1173)
The current solution for titles on input boxes is not obvious to the
user as support varies between bridges. This commit adds an button to
all input boxes with titles in order to make it clear to the user that
additional information is available.
2019-10-31 21:09:44 +01:00
Roliga
c8d5c85c76 formats: Add getMimeType() function (#1299)
Allows getting the expected MIME type of the format's output. A
corresponding MIME_TYPE constant is also defined in FormatAbstract for
the format implementations to overwrite.
2019-10-31 19:00:12 +01:00
somini
d1e4bd7285 [YahtzeeDevDiary] Add new bridge (#1297) 2019-10-31 18:55:08 +01:00
LogMANOriginal
1022b5fdf9
core: Add an option to suppress error reporting (#1179)
Error reporting currently takes place for each error. This can result
in many error messages if a server has connectivity issues (i.e. when
it re-connects to the internet every 24 hours).

This commit adds a new option to the configuration file to define the
number of error reports to suppress before returning an error message
to the user.

Error reports are cached and therefore automatically purged after 24
hours. A successful bridge request does **not** clear the error count
as sporadic issues can be the result of actual problems on the server.

The implementation currently makes no assumption on the type of error,
which means it also suppresses bridge errors in debug mode. The default
value is, however, set to 1 which means all errors are reported.

References #994
2019-10-31 18:49:45 +01:00
LogMANOriginal
e8536ac1b2
core: Add an option to return errors in different formats (#1071)
Bridge errors are currently included as part of the feed to
notify users about erroneous bridges (before that, bridges
silently failed).

This solution, however, can produce a high load of error
messages if servers are down (see #994 for more details).

Admins may also not want to include error messages in feeds
in order to keep those kind of problems away from users or
simply to silently fail by choice.

This commit adds a new configuration section "error" with
one option "output" which can be set to following values:

"feed": To include error messages in the feed (default)
"http": To return a HTTP header for each error
"none": To disable error reporting

Note that errors are always logged to 'error.log' independent
of the settings above.

Closes #1066
2019-10-31 18:40:51 +01:00
Lyra
a0afe36d56 [DownDetectorBridge] Add per-website status fetch. Note that this only fetches the last downtime, as this is the only thing that the API provides. Moreover, the site uses a different ID for every company for every country, resulting in a very large array 2019-10-29 23:14:51 +01:00
Lyra
0b80f9d61c [DownDetectorBridge] Add bridge for DownDetector, and all local variants. Fixes
#1339.
2019-10-29 19:11:28 +01:00
somini
424075981f [EsquerdaNetBridge] Add new Bridge (#1296) 2019-10-29 18:58:12 +01:00
logmanoriginal
c334df91ec composer: Add all details to the composer file
The composer file currently lacks a lot of details, especially the
"name" and "description", but also "require-dev" and "suggest" info.

This commit adds many more details to the composer file and updates
composer.lock for this repository. Technically the project is ready
to be shipped as composer package.
2019-10-28 20:01:19 +01:00
Dominik Thiemermann
f2346fb33e [RevolutBridge] Add new bridge (#1321)
* [RevolutBridge] Add new bridge
2019-10-28 19:49:01 +01:00
Matt DeMoss
8a21fd1476 [BloombergBridge] Remove after site redesign and paywall. (#1238) 2019-10-28 19:27:56 +01:00
somini
2ac44172ac Facebook: Clarify Facebook bridges (#1221)
* Clarify Facebook bridges status

Distinguish between both Facebook bridges by their title.
This preserves all existing URLs.

* Update all URLs to secure HTTPS versions.
* Configure author name abbreviation
* Improve feed names

Use the correct feed name on each bridge.
Make sure the feed names don't repeat the "Facebook" name.
2019-10-28 19:01:04 +01:00
Roliga
4c78721f03 [ParameterValidator] Ensure context has all user provided parameters (#1211)
* [ParameterValidator] Ensure context has all fields

Previously if a bridge had a set of parameters like:

const PARAMETERS = array(
    'ContextA' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        )
    ),
    'ContextB' => array(
        'Param1' => array(
            'name' => 'Param1',
            'required' => true
        ),
        'Param2' => array(
            'name' => 'Param2',
            'required' => true
        )
    )
)

and a query specifying both Param1 and Param2 was provided a 'Mixed
context parameters' error would be returned. This change ensures
ContextA in the above example would not be considered a relevant context.
2019-10-28 17:50:55 +01:00
Christian Archer
04be85996d [BastaBridge] Fix PHP 7.4 crash (#1323)
* Inline the function
2019-10-24 21:57:14 +02:00
Joseph
59be6bded2 [GoogleSearchBridge] Replace 'div[id=ires]' with 'div[id=res]' (#1329) 2019-10-16 21:44:41 +02:00
Joseph
46873e14fe [GoogleSearchBridge] Use getURI() to build URLs (#1330)
* [GoogleSearchBridge] Use getURI() to build URLs
2019-10-16 21:44:28 +02:00
Joseph
0f01cc97a4 [StoriesIGBridge] Add timestamp to feed items (#1331) 2019-10-16 21:44:01 +02:00
Joseph
a70e00a76d [SuperbWallpapersBridge] Delete bridge (#1336) 2019-10-16 21:43:38 +02:00
Joseph
f0260c62c3 [StoriesIGBridge] Use getName() to create custom feed titles (#1332)
* [StoriesIGBridge] Use getName()
2019-10-16 21:41:47 +02:00
Roliga
fc5a1526ca [BandcampBridge] Add band and album feeds (#1317)
* [BandcampBridge] Add band and artist feeds

This can return a limited number of the most recent releases by a band,
or a single release/album. Each release may be given a unique article ID
depending on its track list with the "Releases, new one when track track
changes" option, which should make them show up as new articles when
tracks are added or removed. Releases may also be split up to individual
articles for each track with the "Individual tracks" option.

This uses and undocumented API from the Bandcamp Android app. It's much
faster than loading and parsing the website HTML, and seems to fail less
often with more relaxed rate limits. It's still far from perfect in that
regard though.

The "Individual tracks" option generates requests for each individual
track so that can quickly run into rate limits.

The "Individual tracks" option also has a quirk where tracks released
under e.g. a music label will have their artist set to the label instead
of the actual artist of the track. This is a limitation of the API.
2019-10-16 21:37:25 +02:00
Joseph
4c0e234479 [Bridges] Use HTTPS (#1337)
* [Rule34pahealBridge] Use HTTPS
* [KonachanBridge] Use HTTPS
* [Rule34Bridge] Use HTTPS
* [SafebooruBridge] Use HTTPS
* [TbibBridge] Use HTTPS
* [XbooruBridge] Use HTTPS
* [ScmbBridge] Use HTTPS
* [ReporterreBridge] Use HTTPS
* [BastaBridge] Use HTTPS
* [NiceMatinBridge] Use HTTPS
* [ScoopItBridge] Use HTTPS
* [TheCodingLoveBridge] Use HTTPS
* [Shimmie2Bridge] Use HTTPS
* [HDWallpapersBridge] Use HTTPS
* [GiphyBridge] Use HTTPS
* [PickyWallpapersBridge] Use HTTPS
* [ParuVenduImmoBridge] Use HTTPS
* [ElsevierBridge] Use HTTPS
* [CastorusBridge] Use HTTPS
* [CollegeDeFranceBridge] Use HTTPS
* [MangareaderBridge] Use HTTPS
2019-10-16 21:34:28 +02:00
somini
0eab63d728 Update Facebook URL detection (#1334)
* add detectParameters to FacebookBridge.php
2019-10-16 21:32:29 +02:00
floviolleau
b0884e9158 [VieDeMerdeBridge] Add new bridge for quotes from Vie de Merde (#1313)
* Add new bridge for quotes from Vie de Merde
2019-10-03 22:36:08 +02:00
Nicolas Delsaux
b4581418d4 [PlantUMLBridge] Added bridge for PlantUML. Fixes #1191
* Fixes #1191 by implementing the RSS feed of PlantUML releases
2019-10-03 22:30:22 +02:00
sysadminstory
af1566f40d [ZoneTelechargementBridge] URL and name change (#1302)
Annuaire Telechargement has change name again to go back to Annuaire
Telechargement. Fixes #1279
2019-10-03 22:27:10 +02:00
sysadminstory
529e0d0cca [ExtremeDownloadBridge] Update Website URL (#1303)
Website URL was changed.
2019-10-03 22:25:56 +02:00
Paróczai Olivér
a3532804ac [Readme] Small grammar fixes (#1312) 2019-10-03 22:25:05 +02:00
Anchit Bajaj
8c19146d29 [ListverseBridge - add new bridge (#1305) 2019-10-03 22:24:14 +02:00
Anchit Bajaj
2a3d5865ad [FreeCodeCampBridge] - rss feed for FreeCodeCamp (#1311) 2019-10-03 22:23:14 +02:00
Lyra
4d36c9dc30 Merge branch 'master' of github.com:RSS-Bridge/rss-bridge 2019-10-03 22:14:33 +02:00
Lyra
a2e47a88c3 [InstagramBridge] Add option to get direct links 2019-10-03 22:14:21 +02:00
Anchit Bajaj
b09f50853f [ViceBridge] - RSS feed for Vice Publications. (#1310)
* [ViceBridge] - RSS feed for Vice Publications.
2019-10-03 22:02:30 +02:00
lukasklinger
9b5bf565b3 [N26Bridge] Updated bridge to reflect changes on N26 blog (#1295)
N26 made some changes to their blog, this commit fixes the N26Bridge
2019-10-03 21:58:57 +02:00
Lyra
5cc956367f [core] Fix travis 2019-10-03 21:46:49 +02:00
Lyra
548e28249b [ThePirateBayBridge] Remove nested function 2019-10-03 21:46:24 +02:00
Lyra
684c69b0cd [Releases3DSBridge] Remove nested functions 2019-10-03 21:46:09 +02:00
Lyra
3dae4e0801 [JapanExpoBridge] Remove nested function 2019-10-03 21:45:51 +02:00
Lyra
4622d9be1e [ReadComicsBridge] Deleted bridge since website no longer exists 2019-10-03 21:41:22 +02:00
Nicolas Delsaux
76183dcd44 [GQMagazineBridge] Fix article body detection again (Fixes #1280) 2019-10-03 21:26:41 +02:00
Eugene Molotov
50b234d893 [VkBridge] Photo and timestamp fixes (#1287)
* [VkBridge] Correct parsing of photos, fix timestamp for old posts
2019-09-16 21:30:27 +02:00
Eugene Molotov
af48f36fd2 [VkBridge] Switch maintainer (#1288) 2019-09-16 21:29:45 +02:00
Eugene Molotov
7f6ca23e8f [PikabuBridge] Preserve links (#1286)
* [PikabuBridge] Preserve links
2019-09-16 21:28:41 +02:00
oratosquilla-oratoria
1daef22a3d [NFLRUSBridge] Add new bridge (#1285)
* [NFLRUSBridge] Add new bridge
2019-09-16 21:27:01 +02:00
killruana
c694810d9a [MediapartBridge] Fix article parsing
* Only process article item, fix issue #1292
2019-09-16 21:26:19 +02:00
ORelio
f12f6a2dba [DarkReading] Add DarkReading Bridge (#1289) 2019-09-16 21:25:28 +02:00
Lyra
b1be45df6c
[Configuration] Bump version to dev.2019-09-12 2019-09-12 17:09:30 +02:00
Lyra
b4f393a5cc
[Configuration] Bump version to 2019-07-06 2019-09-12 17:08:15 +02:00
Lyra
29126ebe29
[README] Update list of contributors 2019-09-12 17:07:04 +02:00
triatic
50c971d545 [TwitterBridge] Enable cookies with curl (#1245)
* [TwitterBridge] Enable cookies with curl

Enable cookies in curl, or fall back to `file_get_contents` if in CLI mode with no curl root certificates.
2019-09-12 16:14:48 +02:00