Commit graph

89 commits

Author SHA1 Message Date
Dag
a59793e8d6
refactor: extract CurlHttpClient (#3532)
* refactor: extract CurlHttpClient

* refactor

* interface
2023-07-16 22:07:34 +02:00
Dag
310160fd92
feat: improve http 429 handling (#3541) 2023-07-16 07:18:38 +02:00
Dag
c9a861e259
fix(cache): bug in prior refactor (#3520) 2023-07-09 15:24:29 +02:00
Dag
7881c87bed
fix: various small fixes (#3517)
* fix(asrocknews): Trying to get property src of non-object

Trying to get property 'src' of non-object at bridges/ASRockNewsBridge.php line 37

* refactor(http): tweak max redirs config

* fix(tiktok)

* fix(gizmodo)

* fix(craig)

* fix(nationalg)

* fix(roadandtrack)

* fix(etsy)
2023-07-08 23:21:55 +02:00
Dag
c1c8304fc0
refactor: dont create multiple instances of the cache (#3504) 2023-07-08 17:03:12 +02:00
Dag
5e22459eb6
fix: remove unnecessary calls to purgeCache (#3502) 2023-07-06 18:52:19 +02:00
Dag
caac7f572c
refacor: improve cache interface (#3492)
* fix: proper typehint on setScope

* refactor: type hint setKey()

* typehint
2023-07-06 15:10:30 +02:00
Dag
a9fd3b9e61
fix(CacheInterface): logic bug in getTime (#3491)
* fix(CacheInterface): logic bug in getTime

* test
2023-07-05 17:37:21 +02:00
Dag
748fc9fd65
fix: various small notice fixes (#3474)
* fix(patreon): php notice

* fix(pepperbridge): php notice

* fix(ebay): php notice

* fix(tiktok): php notice

* fix(yandex): fix notice

* fix(justwatch): notice

* lint
2023-07-02 06:40:25 +02:00
Dag
ee498eadf9
fix: move debug mode to config (#3324)
* fix: move debug mode to config

* fix: also move debug_whitelist to .ini config

* fix: move logic back to Debug class

* docs

* docs

* fix: disable debug mode by default

* fix: restore previous behavior for alerts

* fix: center-align alert text
2023-06-02 20:22:09 +02:00
Dag
50865d5741
fix: add additional cloudflare title (Glassdoor specific) (#3342) 2023-05-11 01:33:38 +02:00
Max
c628f99928 use lowercase 2023-05-10 18:40:33 +02:00
Dag
4c3ebb312d
feat: improve error handling ux (#3298)
* feat: improve error handling ux

* feat: add error messages for failed xml parsing
2023-03-20 19:11:51 +01:00
Mynacol
4450e9b973
Let curl select the default HTTP version (#3249)
This essentially reverts b042412416,
as YouTube seems to have fixed their servers.
At least I was able to query the YouTube endpoint around 150 times with
CURL_HTTP_VERSION_2TLS recently. They even advertise HTTP/3 support with
an `alt-svc` HTTP header now.

This unsets CURLOPT_HTTP_VERSION to let curl decide
on the version. This would support all curl versions and opens the
possibility for HTTP/3, but leads to inconsistent behavior depending
on the underlying curl version.

We don't set CURL_HTTP_VERSION_NONE explicitly, as it is always the curl
default and opens the path to let individual bridges override the HTTP
version where necessary.

Alternatively, setting CURL_HTTP_VERSION_2TLS explicitly would lead to consistent behavior
regardless of the curl version, but might uncover old curl bugs before the
developers enabled HTTP/2 by default.
Additionally, that requires at least PHP 7.0.7 (we require PHP 7.4
already) and curl 7.47.0 [1], released on Jan 27 2016 [2].

See also the discussion on https://github.com/RSS-Bridge/rss-bridge/pull/3249

[1] https://www.php.net/manual/curl.constants.php
[2] https://curl.se/docs/releases.html
2023-02-22 17:48:39 +01:00
Dag
a13c4624fb
feat: custom http ua in AO3, fix #3188 (#3189)
* refactor

* feat: custom http ua in AO3, #3188
2022-12-13 09:53:42 +01:00
Dag
a5779d30b5
feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
Dag
2ef98b299f
refactor: extract frontpage to template (#3130)
Also introduce usage of Response object
2022-11-07 18:22:54 +01:00
Dag
e027bd9274
fix: improve FeedExpander (#3103)
* fix: improve FeedExpander

Include the first libxml error in exception.

Give better error message if trying to parse the empty string.

Log all libxml errors if debug mode is enabled.

* error handling and logging tweak
2022-10-29 10:27:02 +02:00
Dag
1f576312ea
feat: in debug mode, include part of http response in exception message (#3090) 2022-10-29 08:42:50 +02:00
Dag
27b3d7c34e
feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Eugene Molotov
bbbd599bc8
fix: do not throw exception on 301-303 http responses (#3014) 2022-09-06 15:40:20 +02:00
Dag
2bbce8ebef
refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag
b042412416
fix: force HTTP 1.1 in curl requests (#2949)
Since curl 7.62.0 the default option is: CURL_HTTP_VERSION_2TLS

Before that the default used to be: CURL_HTTP_VERSION_1_1

Fix #2947
2022-08-05 11:46:32 +02:00
Dag
afcc38786e
fix: use default headers in getContents() (#2927) 2022-07-31 04:21:56 +02:00
Jan Tojnar
5b5f3b4254
Do not use constants for configuration (#2938)
* docs: Do not use constant names when referring to config options

The options are customizable using a config file and no longer hardcoded in index.php since 8ac8e08abf

* Do not use constants for configuration

Since <8ac8e08abf>, they are just set to the configuration object values.
2022-07-24 19:26:12 +02:00
Jan Tojnar
d107f8ed30
Improve Factory variable names (#2895) 2022-07-06 12:14:04 +02:00
Dag
321ec7c8c1
refactor: move cache logic into the factory (#2884) 2022-07-05 13:20:01 +02:00
Dag
4f75591060
Reformat codebase v4 (#2872)
Reformat code base to PSR12

Co-authored-by: rssbridge <noreply@github.com>
2022-07-01 15:10:30 +02:00
Dag
07927008eb
refactor: CacheFactory (#2834) 2022-06-22 18:29:28 +02:00
Jan Tojnar
347f9a3eda
[contents] Add MIME type for mp3 (#2809)
Without this, format tests fail on systems without `/etc/mime.types`.
2022-06-09 22:41:10 -04:00
Dag
cb05cacd6a
fix: add 429 to status codes (#2757) 2022-05-27 15:25:12 +02:00
Dag
5cc34b884a
[core] Improve getContents docs (#2742) 2022-05-22 21:27:23 -04:00
Dag
7d00b0c5df
fix: include http code in exception (#2726) 2022-05-17 23:47:12 +02:00
Dag
0212c4790f
fix: connectivityaction (#2725) 2022-05-17 23:46:37 +02:00
Dag
fcc3707210 refactor: swap the order of sprintf values 2022-05-11 22:37:59 +02:00
Dag
d5e9dbf47d refactor: restore some useful comments 2022-05-11 22:35:03 +02:00
Dag
f5a51038cc fix: error when passing null values
This bug was introduced by me when refactoring the http client.

Fixes:

Fatal error: Uncaught TypeError: Argument 2 passed to getContents() must be of the type array, null given
2022-05-08 04:42:24 +02:00
Dag
5d77d14f9d
feat: add retry logic to the http client (#2692)
* refactor: extract http client

* feat: add retry logic to http client
2022-05-08 03:58:57 +02:00
Patrick Collins
e1c898848f
[contents.php] Fix incorrect reference to UnexpectedResponseException's responseBody. (#2677) 2022-04-23 10:04:56 +05:00
dag
c3a106892d
fix: require curl extension (#2617) 2022-04-10 18:54:18 +02:00
dag
cce11964a4
feat: add a timeout option for http client (#2600) 2022-04-08 21:22:13 +02:00
dag
bed20e9f28
feat: extract curl ua to config value (#2568)
* exclude config.default.ini.php from phpcs
2022-04-03 09:37:39 +02:00
Dag
b6d1c7a58f fix: php notice
Fixes:
Notice: Undefined variable: message in /home/rssbridge/public/lib/contents.php on line 39
2022-03-24 13:52:02 +05:00
Dag
384790537b fix: bug in cloudflare response detection
The cloudflare server header was not recognized in
some cases such as when the server header is "server"
or when the header value is "Cloudflare".
2022-03-24 13:52:02 +05:00
Dag
7bdc53125c fix: properly verify the existence of the curl module
This fixes a bug where it didnt use curl from cli
even though it's installed.

I believe this preserves the original intention to
not require the curl module to be installed.

https://github.com/RSS-Bridge/rss-bridge/pull/979
2022-03-24 13:52:02 +05:00
Paul Staroch
18504f2356
[lib/contents.php] Use variable name 'retVal' instead of 'retval' as variable names are case-sensitive (#2498) 2022-03-14 05:46:30 +05:00
Eugene Molotov
05273a9278
[core] Make getContents exceptions to be handled correctly and correct exception message (#2447)
This commit fixes following issues:
1. 'Unexpected response' error message was returned, even if upstream did not return anything
2. Inability to handle non-20x messages with checking response body
2022-03-12 01:18:01 +05:00
Martin Leyrer
3764348b76
[core] Accept additional "successful" 2xx status codes (#2310) 2021-10-30 01:38:25 +05:00
Thomas
771b851b52
[contents.php] Fix logical error in getSimpleHTMLDOMCached function (#1974)
Previously content was only loaded from cache when debug mode was enabled (the opposite of the expected behavior)
2021-02-09 17:40:16 +05:00
Eugene Molotov
3226a5e31e [core] Use more correct text to indicate errored response from upstream 2020-12-23 17:49:11 +05:00