Commit graph

616 commits

Author SHA1 Message Date
Dag
93620aa105
fix(cache): bug in cache logic (#3553)
It is possible to have a cached item with a very old mtime but it's technically expired.

So, check for presence of time and whether the time it is within 10 days
2023-07-19 22:05:26 +02:00
Dag
6254b8593e
refactor(cache): extract and encapsulate cache expiration logic (#3547)
* refactor(cache): extract and encapsulate cache expiration logic

* fix: logic bug in getSimpleHTMLDOMCached

* fix: silly me, index should of course be on the key column

* silly me again, PRIMARY keys get index by default lol

* comment out the delete portion in loadData

* remove a few log statements

* tweak twitter cache timeout
2023-07-19 05:05:49 +02:00
Dag
08d16322e1
fix: bug in prior conflict merge (#3546) 2023-07-16 22:37:37 +02:00
Dag
a59793e8d6
refactor: extract CurlHttpClient (#3532)
* refactor: extract CurlHttpClient

* refactor

* interface
2023-07-16 22:07:34 +02:00
Dag
310160fd92
feat: improve http 429 handling (#3541) 2023-07-16 07:18:38 +02:00
Dag
eaea8e6640
fix: Undefined index: REMOTE_ADDR in non-web sapi' (#3538) 2023-07-15 21:16:23 +02:00
Fake4d
0f2b55fbef
Update Configuration.php - Old Version Date in new Release (#3526)
Old Version Date in new Release
2023-07-12 11:56:07 +02:00
Dag
69aa751f40
fix(cache): bug in prior refactor (#3525)
* fix(cache): bug in prior refactor

* yup
2023-07-11 23:26:22 +02:00
csisoap
b3bf95bfdd
Replace token for Twitter. (#3522) 2023-07-11 23:17:38 +02:00
Dag
6c0e186d3f
fix(jornaln): Array to string conversion at lib/BridgeAbstract.php li… (#3523)
* fix(jornaln): Array to string conversion at lib/BridgeAbstract.php line 320

* yup
2023-07-11 16:54:59 +02:00
Dag
c9a861e259
fix(cache): bug in prior refactor (#3520) 2023-07-09 15:24:29 +02:00
Dag
dfe78fb379
fix: various small fixes (#3519) 2023-07-09 10:08:30 +02:00
Dag
7881c87bed
fix: various small fixes (#3517)
* fix(asrocknews): Trying to get property src of non-object

Trying to get property 'src' of non-object at bridges/ASRockNewsBridge.php line 37

* refactor(http): tweak max redirs config

* fix(tiktok)

* fix(gizmodo)

* fix(craig)

* fix(nationalg)

* fix(roadandtrack)

* fix(etsy)
2023-07-08 23:21:55 +02:00
Dag
91976f7d56
fix(file cache): acquire lock before writing (#3509) 2023-07-08 17:06:49 +02:00
Dag
c1c8304fc0
refactor: dont create multiple instances of the cache (#3504) 2023-07-08 17:03:12 +02:00
Dag
b594ad2de3
fix: discard empty lines in whitelist.txt (#3507) 2023-07-07 11:25:36 +02:00
Dag
5e22459eb6
fix: remove unnecessary calls to purgeCache (#3502) 2023-07-06 18:52:19 +02:00
Dag
965d7d44c5
feat(sqlite cache): add config options (#3499)
* refactor: sqlite cache

* refactor

* feat: add config options to sqlite cache

* refactor
2023-07-06 15:59:38 +02:00
Dag
caac7f572c
refacor: improve cache interface (#3492)
* fix: proper typehint on setScope

* refactor: type hint setKey()

* typehint
2023-07-06 15:10:30 +02:00
Dag
a9fd3b9e61
fix(CacheInterface): logic bug in getTime (#3491)
* fix(CacheInterface): logic bug in getTime

* test
2023-07-05 17:37:21 +02:00
Dag
a21d496bc7
feat: add default arg to Configuration::getConfig (#3331) 2023-07-05 05:33:22 +02:00
Dag
bf73372d7f
fix: dont be case-sensitive on env vars (#3475) 2023-07-02 06:47:21 +02:00
Dag
748fc9fd65
fix: various small notice fixes (#3474)
* fix(patreon): php notice

* fix(pepperbridge): php notice

* fix(ebay): php notice

* fix(tiktok): php notice

* fix(yandex): fix notice

* fix(justwatch): notice

* lint
2023-07-02 06:40:25 +02:00
Dag
372880b5ef
fix: file cache tweaks (#3470)
* fix: improve file cache

* fix(filecache): log when unserialize fails
2023-06-30 22:31:19 +02:00
Dag
8eabdbe5f8
fix(Twitter): properly find time line entries when ordering is inconsistent (#3461) 2023-06-27 16:11:41 +02:00
Thomas
54d626d5cd
[XPathAbstract] Use baseURI to fix relative links (if available) (#3439) 2023-06-17 17:53:00 +02:00
Dag
0a8fe57003
feat: enable bridges using env var (#3428)
* refactor: bridgefactory, add tests

* refactor: move defaultly enabled bridges to config

* refactor

* refactor

* feat: add support for enabling bridges with env var
2023-06-11 03:16:03 +02:00
Dag
fbaf26e8bf
fix(html_format): add spacing below date if author is missing (#3425)
* small ui tweak

* remove unused <div>

* refactor: rename method

* refactor: inline const

* refactor
2023-06-08 23:04:16 +02:00
Dag
ee498eadf9
fix: move debug mode to config (#3324)
* fix: move debug mode to config

* fix: also move debug_whitelist to .ini config

* fix: move logic back to Debug class

* docs

* docs

* fix: disable debug mode by default

* fix: restore previous behavior for alerts

* fix: center-align alert text
2023-06-02 20:22:09 +02:00
mrnoname1000
87b9f2dd94
[core] Fix XPathAbstract while working around Simple HTML DOM bug (#3408) 2023-05-21 21:06:35 +02:00
Dag
f803ffa79a
fix: ArgumentCountError: DOMDocument::getElementsByTagName() expects exactly 1 argument, 2 given, #3406 (#3407) 2023-05-21 19:59:39 +02:00
mrnoname1000
3e0d024888
[core] Fix defaultLinkTo for simple_html_dom objects (#3404) 2023-05-20 00:02:17 +02:00
Dag
cfe81ab2ac
fix: Call to a member function setAttribute() on int, #3402 (#3403) 2023-05-19 16:05:52 +02:00
mrnoname1000
096c3bca73
[XPathAbstract] Fix relative links in fetched HTML (#3401)
* [core] Make defaultLinkTo compatible with DOMDocument

* [XPathAbstract] Fix relative links in fetched HTML
2023-05-18 13:50:50 +02:00
mrnoname1000
d0f7f5e2d8
[New Bridge] FiderBridge (#3378)
* [core] Add config parameter to markdownToHtml

* [FiderBridge] New bridge
2023-05-11 21:24:12 +02:00
Dag
50865d5741
fix: add additional cloudflare title (Glassdoor specific) (#3342) 2023-05-11 01:33:38 +02:00
mrnoname1000
c6c4b3a24f
[XPathAbstract] Fix encoding on feed title (#3365) 2023-05-11 01:32:01 +02:00
mrnoname1000
63dc500ae0
[XPathAbstract] Save HTML for entry content (#3366) 2023-05-11 01:31:34 +02:00
Dag
ff49c9f731
fix(TwitterBridge): repair fetching of tweets by username (#3385)
* feat: alpha version of new twitter bridge

* fix: refetch guest_token if expired

* fix: purge cache

* fix: safeguards

* fix

* fix: two notices

* fix

* fix: use factory to create cache

* fix: fail properly instead of die()
2023-05-10 21:45:44 +02:00
Max
c628f99928 use lowercase 2023-05-10 18:40:33 +02:00
Eugene Molotov
343fd36671
[core] Remove hardcoded maximum duration of 24 hours in loadCacheValue (#3355) 2023-04-19 17:53:35 +02:00
Dag
9bb04ba848
Prepare 2023-03-21 release (#3323)
* fix: upgrade version string in php code
2023-03-22 19:32:19 +01:00
Dag
4c3ebb312d
feat: improve error handling ux (#3298)
* feat: improve error handling ux

* feat: add error messages for failed xml parsing
2023-03-20 19:11:51 +01:00
Dag
9e9a697b8b
feat: add config option "path" for file cache (#3297) 2023-03-20 19:10:01 +01:00
Dag
058e792b8f
feat: add filecache config to enable/disable real purge (#3263)
* refactor: cachefactory

* feat: add filecache config to enable/disable real purge

* test: fix test
2023-03-06 21:50:40 +01:00
Dag
007f2b2d8a
feat: sanitize root folder also in php error messages (#3262) 2023-03-06 21:47:25 +01:00
Dag
a01c1f6ab0
fix: disallow usage of default password (#3284) 2023-03-06 20:43:44 +01:00
Bocki
f0e5ef0fc5
[Various] getKey replacements and docu (#3283)
* [Various] getKey replacements and docu

* more bridges and fix to the abstract

* linting

* revert bandcampdaily. doing more than i thought
2023-03-06 20:01:51 +01:00
Bocki
f3f98a117c
[Core] Add getKey function (#3275)
* [Core] Add getKey function
2023-03-02 13:25:57 +01:00
Mynacol
4450e9b973
Let curl select the default HTTP version (#3249)
This essentially reverts b042412416,
as YouTube seems to have fixed their servers.
At least I was able to query the YouTube endpoint around 150 times with
CURL_HTTP_VERSION_2TLS recently. They even advertise HTTP/3 support with
an `alt-svc` HTTP header now.

This unsets CURLOPT_HTTP_VERSION to let curl decide
on the version. This would support all curl versions and opens the
possibility for HTTP/3, but leads to inconsistent behavior depending
on the underlying curl version.

We don't set CURL_HTTP_VERSION_NONE explicitly, as it is always the curl
default and opens the path to let individual bridges override the HTTP
version where necessary.

Alternatively, setting CURL_HTTP_VERSION_2TLS explicitly would lead to consistent behavior
regardless of the curl version, but might uncover old curl bugs before the
developers enabled HTTP/2 by default.
Additionally, that requires at least PHP 7.0.7 (we require PHP 7.4
already) and curl 7.47.0 [1], released on Jan 27 2016 [2].

See also the discussion on https://github.com/RSS-Bridge/rss-bridge/pull/3249

[1] https://www.php.net/manual/curl.constants.php
[2] https://curl.se/docs/releases.html
2023-02-22 17:48:39 +01:00
Dag
c27a300e02
test: add failing mastodon test (#3255)
* fix: refactor cache factory

* test: add failing test

* add null cache
2023-02-15 21:22:37 +01:00
Dag
7f1b32f390
feat: add a proper feed item uid when the bridge errors out (#3237)
* refactor: move function to class

* fix: use the computed bridge name as cache key

* refactor: extract method

* fix: set a feed item uid on errors

* docs

* fix: remove year from uid
2023-02-02 22:53:01 +01:00
Dag
a13c4624fb
feat: custom http ua in AO3, fix #3188 (#3189)
* refactor

* feat: custom http ua in AO3, #3188
2022-12-13 09:53:42 +01:00
ORelio
d592e2cb15
[Core] Add html/convertLazyLoading (+ document stripRecursiveHTMLSection) (#3157)
* [core] Add html/convertLazyLoading($dom)

Looks for lazy-loading attributes such as 'data-src' and converts
them back to regular ones such as 'src', easier for RSS readers.
It also converts <picture> elements to plain <img> elements.

* [core] Document html/stripRecursiveHTMLSection()

Add documentation for that function (no code changes).

* [WordPressBridge] Use convertLazyLoading()

* [WordPressBridge] Unwrap image figures

<img> inside <figure> may not display on RSS readers.
This converts them back to <img>, without losing caption if present.

* [ZDNet] Convert lazy loading images

* [code] html/stripRecursiveHTMLSection: Fix typo
2022-11-20 12:41:59 +01:00
Eugene Molotov
8990aeb9f4
[core] Merge GET and POST args into request array (#3155) 2022-11-20 12:41:20 +01:00
Dag
88766e6fde
fix: produce smaller log records for http exceptions (#3153) 2022-11-18 21:36:06 +01:00
Dag
a5779d30b5
feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
Dag
dbab225fd2
fix: Call to a member function find() on bool (#3146)
* fix: Call to a member function find() on bool

Happens when defaultLinkTo() is passed the empty string.

* fix: prevent exception in defaultLinkTo() when passed the empty string

* refactor
2022-11-15 03:01:27 +01:00
Dag
001427243c
feat: system alert message (#3139) 2022-11-15 00:32:04 +01:00
Dag
95c199c2eb
fix: various php notices (#3145)
* fix: notice

* fix: Trying to get property content of non-object at bridges/PcGamerBridge.php line 36

* fix: better exception message

* fix: strpos(): Non-string needles will be interpreted as strings in the future. Use an explicit chr() call to preserve the current behavior
2022-11-15 00:30:51 +01:00
Dag
2ef98b299f
refactor: extract frontpage to template (#3130)
Also introduce usage of Response object
2022-11-07 18:22:54 +01:00
Dag
23f8c81646
refactor/fix: css organization and error rendering (#3117)
* fix: php notice

* refactor/feat: merge HtmlFormat.css into style.css

Also improve ux of error rendering.

* fix: center-align footer text
2022-10-29 10:46:37 +02:00
Dag
e027bd9274
fix: improve FeedExpander (#3103)
* fix: improve FeedExpander

Include the first libxml error in exception.

Give better error message if trying to parse the empty string.

Log all libxml errors if debug mode is enabled.

* error handling and logging tweak
2022-10-29 10:27:02 +02:00
Dag
1f576312ea
feat: in debug mode, include part of http response in exception message (#3090) 2022-10-29 08:42:50 +02:00
Dag
52af2ae34c
fix: php errors (notices) (#3115) 2022-10-26 00:47:45 +02:00
ORelio
05f2fb5ec7
[FeedExpander] Decode HTML entities in title (#3110)
Feed item title may contain HTML entities that we need to decode,
else they are encoded twice when generating the expanded feed.
2022-10-20 18:26:43 +02:00
Eugene Molotov
d483bf2b81
[core] Implement bearer token authentication (#3043) 2022-10-19 18:39:35 +02:00
Dag
ffbc107687
Improve logging and error handling (#3059)
* refactor: logging and errror handling
2022-10-16 17:55:43 +02:00
Dag
e21394d2d3
refactor: html format (#3062)
* refactor: html format

Fix a few small bugs too

* fix

* fix

* trigger build

* striptags instead of encode title
2022-10-16 12:03:57 +02:00
Dag
aacba5b1a8
fix: too strict url validation in feed item (#3058)
Urls such as https://example.com/réponse were rejected

Fix https://github.com/RSS-Bridge/rss-bridge/issues/3018#issuecomment-1254159203
2022-09-21 23:07:56 +02:00
Eugene Molotov
3d9fead463
[ActionFactory] Allow camel-case action names (#3044)
Dash symbol is used to convert dash-seperated string to camel-cased string
2022-09-20 18:17:57 +02:00
Dag
2d272117cc
fix: dont error out for invalid env names (#3030) 2022-09-12 23:14:11 +02:00
Dag
94ae098ef5
fix: various fixes (#3023)
* improve twitch error message

* fix worldcosplay notice

* fix: add new video image to telegram

* fix: reuters

* fix: formula1

* twitter
2022-09-09 20:18:50 +02:00
Dag
27b3d7c34e
feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Eugene Molotov
bbbd599bc8
fix: do not throw exception on 301-303 http responses (#3014) 2022-09-06 15:40:20 +02:00
Dag
8ea9472300
feat: improve exception message when xml parsing fails (#3009) 2022-09-05 14:26:11 +02:00
Dag
5e09a14acc
fix: include git tag in version (#3000)
Also include os and php version in github issue template.
2022-09-04 07:21:57 +02:00
Dag
693d6edfbf
fix: php7.3 parser error in Configuration.php (#2999)
Dont fail a basic lint so that the proper error message can be shown when on unsupported versions.
2022-09-04 04:50:01 +02:00
Dag
57d5aa45f7
fix: php notice in eztvbridge (#2998)
* fix: php notice in eztvbridge

Fixes Undefined property: stdClass::$torrents

* lint
2022-09-04 04:35:21 +02:00
Dag
f40ed566be
fix: absolute urls for images in pixivbridge (#2993) 2022-09-02 20:35:17 +02:00
Dag
3c2353c0ec
fix: bug in previous refactor (#2992)
fix #2991
2022-08-31 18:16:19 +02:00
Dag
97808abca1
refactor: rename rssbridge.php to bootstrap.php (#2987)
Fix #2986
2022-08-27 23:01:06 +02:00
Dag
5165ea265d
fix: case-insensitive config from env, fix #2935 (#2973)
* refactor

* fix: case-sensitive config from env, fix #2935

* lowercase all config section and keys

* test: add test for case-insensitivity
2022-08-23 21:19:53 +02:00
Dag
0de1694371
refactor: extract index.php to RssBridge.php (#2961)
* refactor: extract entry point into class

* refactor: js

* refactor: extract frontpage action
2022-08-18 22:52:01 +02:00
Dag
502799a74c
feat: use bridge description and short name in search (#2952)
* refactor: search.js

* feat: use bridge description and short name in search

* fix bug in previous merge commit

Also reformat string from tabs to spaces
2022-08-06 23:12:30 +02:00
Dag
2bbce8ebef
refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag
b042412416
fix: force HTTP 1.1 in curl requests (#2949)
Since curl 7.62.0 the default option is: CURL_HTTP_VERSION_2TLS

Before that the default used to be: CURL_HTTP_VERSION_1_1

Fix #2947
2022-08-05 11:46:32 +02:00
Dag
ecb486794b
refactor: use static values for cache scope
This fixes a future problem when code is placed under a namespace because `get_class($bridge)` will then return e.g. `RssBridge\Bridge\TwitterBridge` instead of the the current value `TwitterBridge`.

Also a bit refactoring of `Configuration.php`.
2022-08-02 15:03:54 +02:00
Dag
afcc38786e
fix: use default headers in getContents() (#2927) 2022-07-31 04:21:56 +02:00
Dag
0a060b2ad6
[Gab] feat: add new bridge GabBridge (#2920) 2022-07-31 03:52:27 +02:00
Jan Tojnar
5b5f3b4254
Do not use constants for configuration (#2938)
* docs: Do not use constant names when referring to config options

The options are customizable using a config file and no longer hardcoded in index.php since 8ac8e08abf

* Do not use constants for configuration

Since <8ac8e08abf>, they are just set to the configuration object values.
2022-07-24 19:26:12 +02:00
Dag
5e52ecc3f8
test: add new test for Configuration (#2915) 2022-07-10 20:05:27 +02:00
Dag
a966213cd7
refactor: inject the action params via its execute method (#2907) 2022-07-08 21:06:14 +02:00
Dag
abc4af43b3
feat: improve error handling (#2902) 2022-07-08 20:39:13 +02:00
Dag
abfc6b4633
feat: introduce template engine (#2899) 2022-07-08 14:17:25 +02:00
Jan Tojnar
951092eef3
Fix coding style missed by phpbcf (#2901)
$ composer require --dev friendsofphp/php-cs-fixer

$ echo >.php-cs-fixer.dist.php "<?php

$finder = PhpCsFixer\Finder::create()
    ->in(__DIR__);

$rules = [
    '@PSR12' => true,
    // '@PSR12:risky' => true,
    '@PHP74Migration' => true,
    // '@PHP74Migration:risky' => true,
    // buggy, duplicates existing comment sometimes
    'no_break_comment' => false,
    'array_syntax' => true,
    'lowercase_static_reference' => true,
    'visibility_required' => false,
    // Too much noise
    'binary_operator_spaces' => false,
    'heredoc_indentation' => false,
    'trailing_comma_in_multiline' => false,
];

$config = new PhpCsFixer\Config();

return $config
    ->setRules($rules)
    // ->setRiskyAllowed(true)
    ->setFinder($finder);

"

$ vendor/bin/php-cs-fixer --version
PHP CS Fixer 3.8.0 BerSzcz against war! by Fabien Potencier and Dariusz Ruminski.
PHP runtime: 8.1.7

$ vendor/bin/php-cs-fixer fix
$ rm .php-cs-fixer.cache
$ vendor/bin/php-cs-fixer fix
2022-07-08 13:00:52 +02:00
Jan Tojnar
dbf8c5b7ae
refactor(BridgeFactory): make methods only accept valid class names (#2897)
This moves the responsibility for getting a valid class name
to the users of BridgeFactory, avoiding the repeated sanitation.
Improper use can also be checked statically.
2022-07-08 12:54:23 +02:00
Jan Tojnar
d107f8ed30
Improve Factory variable names (#2895) 2022-07-06 12:14:04 +02:00
Jan Tojnar
ab6aca3163 lib/Configuration: Remove redundant comment
It was just getting out of sync:

- Minimum PHP version was bumped in 8365a7a34d
- Cache directory permission check was removed in 8e2b65556f
- Whitelist permission check was removed in d4e867f240
2022-07-06 03:34:37 +02:00
Dag
321ec7c8c1
refactor: move cache logic into the factory (#2884) 2022-07-05 13:20:01 +02:00