Commit graph

79 commits

Author SHA1 Message Date
Dag
50865d5741
fix: add additional cloudflare title (Glassdoor specific) (#3342) 2023-05-11 01:33:38 +02:00
Max
c628f99928 use lowercase 2023-05-10 18:40:33 +02:00
Dag
4c3ebb312d
feat: improve error handling ux (#3298)
* feat: improve error handling ux

* feat: add error messages for failed xml parsing
2023-03-20 19:11:51 +01:00
Mynacol
4450e9b973
Let curl select the default HTTP version (#3249)
This essentially reverts b042412416,
as YouTube seems to have fixed their servers.
At least I was able to query the YouTube endpoint around 150 times with
CURL_HTTP_VERSION_2TLS recently. They even advertise HTTP/3 support with
an `alt-svc` HTTP header now.

This unsets CURLOPT_HTTP_VERSION to let curl decide
on the version. This would support all curl versions and opens the
possibility for HTTP/3, but leads to inconsistent behavior depending
on the underlying curl version.

We don't set CURL_HTTP_VERSION_NONE explicitly, as it is always the curl
default and opens the path to let individual bridges override the HTTP
version where necessary.

Alternatively, setting CURL_HTTP_VERSION_2TLS explicitly would lead to consistent behavior
regardless of the curl version, but might uncover old curl bugs before the
developers enabled HTTP/2 by default.
Additionally, that requires at least PHP 7.0.7 (we require PHP 7.4
already) and curl 7.47.0 [1], released on Jan 27 2016 [2].

See also the discussion on https://github.com/RSS-Bridge/rss-bridge/pull/3249

[1] https://www.php.net/manual/curl.constants.php
[2] https://curl.se/docs/releases.html
2023-02-22 17:48:39 +01:00
Dag
a13c4624fb
feat: custom http ua in AO3, fix #3188 (#3189)
* refactor

* feat: custom http ua in AO3, #3188
2022-12-13 09:53:42 +01:00
Dag
a5779d30b5
feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
Dag
2ef98b299f
refactor: extract frontpage to template (#3130)
Also introduce usage of Response object
2022-11-07 18:22:54 +01:00
Dag
e027bd9274
fix: improve FeedExpander (#3103)
* fix: improve FeedExpander

Include the first libxml error in exception.

Give better error message if trying to parse the empty string.

Log all libxml errors if debug mode is enabled.

* error handling and logging tweak
2022-10-29 10:27:02 +02:00
Dag
1f576312ea
feat: in debug mode, include part of http response in exception message (#3090) 2022-10-29 08:42:50 +02:00
Dag
27b3d7c34e
feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Eugene Molotov
bbbd599bc8
fix: do not throw exception on 301-303 http responses (#3014) 2022-09-06 15:40:20 +02:00
Dag
2bbce8ebef
refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag
b042412416
fix: force HTTP 1.1 in curl requests (#2949)
Since curl 7.62.0 the default option is: CURL_HTTP_VERSION_2TLS

Before that the default used to be: CURL_HTTP_VERSION_1_1

Fix #2947
2022-08-05 11:46:32 +02:00
Dag
afcc38786e
fix: use default headers in getContents() (#2927) 2022-07-31 04:21:56 +02:00
Jan Tojnar
5b5f3b4254
Do not use constants for configuration (#2938)
* docs: Do not use constant names when referring to config options

The options are customizable using a config file and no longer hardcoded in index.php since 8ac8e08abf

* Do not use constants for configuration

Since <8ac8e08abf>, they are just set to the configuration object values.
2022-07-24 19:26:12 +02:00
Jan Tojnar
d107f8ed30
Improve Factory variable names (#2895) 2022-07-06 12:14:04 +02:00
Dag
321ec7c8c1
refactor: move cache logic into the factory (#2884) 2022-07-05 13:20:01 +02:00
Dag
4f75591060
Reformat codebase v4 (#2872)
Reformat code base to PSR12

Co-authored-by: rssbridge <noreply@github.com>
2022-07-01 15:10:30 +02:00
Dag
07927008eb
refactor: CacheFactory (#2834) 2022-06-22 18:29:28 +02:00
Jan Tojnar
347f9a3eda
[contents] Add MIME type for mp3 (#2809)
Without this, format tests fail on systems without `/etc/mime.types`.
2022-06-09 22:41:10 -04:00
Dag
cb05cacd6a
fix: add 429 to status codes (#2757) 2022-05-27 15:25:12 +02:00
Dag
5cc34b884a
[core] Improve getContents docs (#2742) 2022-05-22 21:27:23 -04:00
Dag
7d00b0c5df
fix: include http code in exception (#2726) 2022-05-17 23:47:12 +02:00
Dag
0212c4790f
fix: connectivityaction (#2725) 2022-05-17 23:46:37 +02:00
Dag
fcc3707210 refactor: swap the order of sprintf values 2022-05-11 22:37:59 +02:00
Dag
d5e9dbf47d refactor: restore some useful comments 2022-05-11 22:35:03 +02:00
Dag
f5a51038cc fix: error when passing null values
This bug was introduced by me when refactoring the http client.

Fixes:

Fatal error: Uncaught TypeError: Argument 2 passed to getContents() must be of the type array, null given
2022-05-08 04:42:24 +02:00
Dag
5d77d14f9d
feat: add retry logic to the http client (#2692)
* refactor: extract http client

* feat: add retry logic to http client
2022-05-08 03:58:57 +02:00
Patrick Collins
e1c898848f
[contents.php] Fix incorrect reference to UnexpectedResponseException's responseBody. (#2677) 2022-04-23 10:04:56 +05:00
dag
c3a106892d
fix: require curl extension (#2617) 2022-04-10 18:54:18 +02:00
dag
cce11964a4
feat: add a timeout option for http client (#2600) 2022-04-08 21:22:13 +02:00
dag
bed20e9f28
feat: extract curl ua to config value (#2568)
* exclude config.default.ini.php from phpcs
2022-04-03 09:37:39 +02:00
Dag
b6d1c7a58f fix: php notice
Fixes:
Notice: Undefined variable: message in /home/rssbridge/public/lib/contents.php on line 39
2022-03-24 13:52:02 +05:00
Dag
384790537b fix: bug in cloudflare response detection
The cloudflare server header was not recognized in
some cases such as when the server header is "server"
or when the header value is "Cloudflare".
2022-03-24 13:52:02 +05:00
Dag
7bdc53125c fix: properly verify the existence of the curl module
This fixes a bug where it didnt use curl from cli
even though it's installed.

I believe this preserves the original intention to
not require the curl module to be installed.

https://github.com/RSS-Bridge/rss-bridge/pull/979
2022-03-24 13:52:02 +05:00
Paul Staroch
18504f2356
[lib/contents.php] Use variable name 'retVal' instead of 'retval' as variable names are case-sensitive (#2498) 2022-03-14 05:46:30 +05:00
Eugene Molotov
05273a9278
[core] Make getContents exceptions to be handled correctly and correct exception message (#2447)
This commit fixes following issues:
1. 'Unexpected response' error message was returned, even if upstream did not return anything
2. Inability to handle non-20x messages with checking response body
2022-03-12 01:18:01 +05:00
Martin Leyrer
3764348b76
[core] Accept additional "successful" 2xx status codes (#2310) 2021-10-30 01:38:25 +05:00
Thomas
771b851b52
[contents.php] Fix logical error in getSimpleHTMLDOMCached function (#1974)
Previously content was only loaded from cache when debug mode was enabled (the opposite of the expected behavior)
2021-02-09 17:40:16 +05:00
Eugene Molotov
3226a5e31e [core] Use more correct text to indicate errored response from upstream 2020-12-23 17:49:11 +05:00
Joshua Coales
d05a8b79fe
[contents.php] Fix return type hints (#1824) 2020-11-08 12:19:18 +05:00
triatic
3ad126cdf2
[core] Add headers to file_get_contents (#1623)
Add response headers to file_get_contents() method. Headers are used by some bridges.
2020-06-25 12:22:05 +02:00
logmanoriginal
3bc8c9468a phpcs: Always use long array syntax
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.

All failures have been fixed.
2019-11-01 18:06:55 +01:00
LogMANOriginal
cdc1d9c9ba
action: Add action to check bridge connectivity (#1147)
* action: Add action to check bridge connectivity

It is currently not simply possible to check if the remote
server for a bridge is reachable or not, which means some
of the bridges might no longer work because the server is
no longer on the internet.

In order to find those bridges we can either check each
bridge individually (which takes a lot of effort), or use
an automated script to do this for us.

If a server is no longer reachable it could mean that it is
temporarily unavailable, or shutdown permanently. The results
of this script will at least help identifying such servers.

* [Connectivity] Use Bootstrap container to properly display contents

* [Connectivity] Limit connectivity checks to debug mode

Connectivity checks take a long time to execute and can require a lot
of bandwidth. Therefore, administrators should be able to determine
when and who is able to utilize this action. The best way to prevent
regular users from accessing this action is by making it available in
debug mode only (public servers should never run in debug mode anyway).

* [Connectivity] Split implemenation into multiple files

* [Connectivity] Make web page responsive to user input

* [Connectivity] Make status message sticky

* [Connectivity] Add icon to the status message

* [contents] Add the ability for getContents to return header information

* [Connectivity] Add header information to the reply Json data

* [Connectivity] Add new status (blue) for redirected sites

Also adds titles to status icons (Successful, Redirected, Inactive, Failed)

* [Connectivity] Fix show doesn't work for inactive bridges

* [Connectivity] Fix typo

* [Connectivity] Catch errors in promise chains

* [Connectivity] Allow search by status and update dynamically

* [Connectivity] Add a progress bar

* [Connectivity] Use bridge factory

* [Connectivity] Import Bootstrap v4.3.1 CSS
2019-10-31 22:02:38 +01:00
triatic
b6be18d585 [contents] Respect passed headers for file_get_contents() (#1234)
* [contents] Respect passed headers for file_get_contents()
2019-07-29 12:05:13 +02:00
logmanoriginal
2460b67886 cache: Refactor cache factory to non-static class
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.

References #1001
2019-06-18 19:04:19 +02:00
logmanoriginal
70542686bb [contents] Fix parsing of incomplete headers
Response headers may contain fields with no values.

Example:
  "Referrer-Policy: "

In this case the current implementation of explode() results in an
error because there is no content after ": ". Changing the delimiter
to ":" and trimming the value manually fixes that issue.
2019-06-09 17:18:08 +02:00
fulmeek
21d3bf3b60 caches: Refactor the API (#1060)
- For consistency, functions should always return null on non-existing data.

- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.

- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.

- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
2019-04-29 20:12:43 +02:00
logmanoriginal
88aae6fd95 core: Apply changes to fix broken Travis builds
Travis-CI recently got updated, which causes existing builds to fail.
For example: https://travis-ci.org/RSS-Bridge/rss-bridge/builds/507568117

Indenting multi-line arguments of functions fixes it.
2019-03-20 19:23:22 +01:00
logmanoriginal
556a417dd6 core: Add support for custom cache types via config.ini.php
This commit adds support for a new parameter which specifies the type
of cache to use for caching. It is specified in config.ini.php:

 [cache]

 type = "..."

Currently only one type of cache is supported (see /caches). All uses
of 'FileCache' were replaced by this configuration option.

Note: Caching currently depends on files and folders (due to FileCache).
Experience may vary depending on the selected cache type. For now always
check if FileCache is working before testing alternative types.

References #1000
2019-02-06 18:52:44 +01:00