Commit graph

47 commits

Author SHA1 Message Date
Dag
2e5d2a88f3
fix: only escape iframe,script and link for html output (#3882) 2024-01-09 21:36:42 +01:00
Dag
3ce94409ab
feat: support itunes namespace in top channel feed (#3776)
Also preserves other properties.
2024-01-09 20:18:33 +01:00
Teemu Ikonen
658391263e
Add 'itunes:duration' tag for items with duration (#3774)
* [{Atom,Mrss}Format] Allow itunes tags on items without enclosure

* [Arte7Bridge] Add $item['itunes']['duration'] value
2023-10-19 17:02:53 +02:00
Dag
ef5bd83bd0
feat: preserve and reproduce podcast feeds (itunes rss module) (#3759) 2023-10-16 02:58:03 +02:00
Dag
f421c45b21
test: add feed item test (#3709)
* test: add feed item test

also some refactor

* yup

* yup
2023-09-25 22:32:15 +02:00
Dag
fbaf26e8bf
fix(html_format): add spacing below date if author is missing (#3425)
* small ui tweak

* remove unused <div>

* refactor: rename method

* refactor: inline const

* refactor
2023-06-08 23:04:16 +02:00
Dag
27b3d7c34e
feat: improve logging and error handling (#2994)
* feat: improve logging and error handling

* trim absolute path from file name

* fix: suppress php errors from xml parsing

* fix: respect the error reporting level in the custom error handler

* feat: dont log error which is produced by bots

* ignore error about invalid bridge name

* upgrade bridge exception from warning to error

* remove remnants of using phps builin error handler

* move responsibility of printing php error from logger to error handler

* feat: include url in log record context

* fix: always include url in log record contect

Also ignore more non-interesting exceptions.

* more verbose httpexception

* fix

* fix
2022-09-08 19:07:57 +02:00
Dag
2bbce8ebef
refactor: general code base refactor (#2950)
* refactor

* fix: bug in previous refactor

* chore: exclude phpcompat sniff due to bug in phpcompat

* fix: do not leak absolute paths

* refactor/fix: batch extensions checking, fix DOS issue
2022-08-06 22:46:28 +02:00
Dag
4f75591060
Reformat codebase v4 (#2872)
Reformat code base to PSR12

Co-authored-by: rssbridge <noreply@github.com>
2022-07-01 15:10:30 +02:00
Jan Tojnar
10eb1c9a95
[FormatAbstract]: Ensure sanitizeHtml is given string (#2791)
Sometimes `Item::getContent` returns `null`, in which case `sanitizeHtml`
would pass it to `str_replace`, which would raise `E_DEPRECATED` on PHP 8.1.
2022-06-17 20:46:15 +02:00
Jan Tojnar
1af6cbeb1e
[XML formats] Ensure elements are connected to DOM before further manipulation (#2806)
We are setting xmlns attributes at the root element but PHP would
still attach redundant ones to the DOM elements created with `createElementNS`.
That was because PHP reconciles namespace attributes when appending elements to DOM
but since we previously only attached the elements after all children were attached,
the reconciliation algorithm was not able to see the root element’s attributes.

To fix this, let’s attach each element to its parent immediately after it is created.
2022-06-09 18:33:23 +02:00
Jan Tojnar
90d22f0d80
[{Atom,Mrss}Format]: Generate using DomDocument (#2771)
* [AtomFormat]: Generate using DomDocument

This will escape the HTML content for us as needed.

* [MrssFormat]: Generate using DomDocument

This will escape the HTML content for us as needed.
2022-06-07 23:22:03 +02:00
Jan Tojnar
fb501652d5
Formats: Remove display & related method (#2776)
Format should not be responsible for sending HTTP response.
2022-06-07 18:05:33 +02:00
Roliga
c8d5c85c76 formats: Add getMimeType() function (#1299)
Allows getting the expected MIME type of the format's output. A
corresponding MIME_TYPE constant is also defined in FormatAbstract for
the format implementations to overwrite.
2019-10-31 19:00:12 +01:00
fulmeek
80f6a8b3d4 [MrssFormat] Rework to make it valid RSS 2.0 + Media RSS (#996) 2019-02-06 17:18:33 +01:00
LogMANOriginal
988635dcf3
core: Add FeedItem class (#940)
Add transformation from legacy items to FeedItems, before transforming
items to the desired format. This allows using legacy bridges alongside
bridges that return FeedItems.

As discussed in #940, instead of throwing exceptions on invalid
parameters, add messages to the debug log instead

Add support for strings to setTimestamp(). If the provided timestamp
is a string, automatically try to parse it using strtotime().

This allows bridges to simply use `$item['timestamp'] = $timestamp;`
instead of `$item['timestamp'] = strtotime($timestamp);`

Support simple_html_dom_node as input paramter for setURI

Support simple_html_dom_node as input parameter for setContent
2018-12-26 22:41:32 +01:00
logmanoriginal
e7d3a006c8 global: Fix code violations 2018-12-26 21:58:07 +01:00
logmanoriginal
946e66e9df core: Use REPOSITORY constant where applicable 2018-11-05 19:05:59 +01:00
logmanoriginal
392e3ff6c7 phpcs: Fix violations 2018-11-05 12:55:58 +01:00
triatic
85e8a67568 [MrssFormat.php] Prevent PHP Notice (#858)
Prevent PHP Notice when running in CLI mode
2018-10-15 18:14:06 +02:00
Matthew Seal
e34ef6cb4f [MrssFormat] Escape double quotes in XML attributes (#813)
XML attributes need to have certain characters escaped to be valid. The title attribute can have double quotes in it which need to be properly encoded for attributes.
2018-09-15 14:13:05 +02:00
ORelio
de8cee6a1c Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802)
* Debug mode improvements

 - Improve debug warning message
 - Restore error reporting in debug mode
 - Fix 'notice' messages for unset fields

* Add parsing utility functions

html.php
 - extractFromDelimiters
 - stripWithDelimiters
 - stripRecursiveHTMLSection
 - markdownToHtml (partial)

bridges
 - remove now-duplicate functions
 - call functions from html.php instead

* [Anidex] New bridge

Anime torrent tracker

* [Anime-Ultime] Restore thumbnail

* [CNET] Recreate bridge

Full rewrite as the previous one was broken

* [Dilbert] Minor URI fix

Use new self::URI property

* [EstCeQuonMetEnProd] Fix content extraction

Bridge was broken

* [Facebook] Fix "SpSonsSoriSsés" label

... which was taking space in item title

* [Futura-Sciences] Use HTTPS, More cleanup

Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements

* [GBATemp] Multiple fixes

- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached

* [JapanExpo] Fix bridge, HTTPS, thumbnails

- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures

* [LeMondeInformatique] Fix bridge, HTTPS

- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue

* [Nextgov] Fix content extraction

- Restore thumbnail and use small image
- Field extraction fixes

* [NextInpact] Add categories and filtering by type

- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once

* [NyaaTorrents] New bridge

Anime torrent tracker

* [Releases3DS] Cache content, restore thumbnail

- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure

* [TheHackerNews] Fix bridge

 - Fix content extraction including article body
 - Restore thumbnail as enclosure

* [WeLiveSecurity] HTTPS, Fix content extraction

- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body

* [WordPress] Reduce timeout, more content selectors

- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup

* [YGGTorrent] Increase limit, use cache

- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached

* [ZDNet] Rewrite with FeedExpander

- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body

* [Main] Handle MIME type for enclosures

Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).

One can force enclosure type using #.ext anchor (hacky, needs improving)

* [FeedExpander] Improve field extraction

- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages

* [Pull] Coding style fixes for #802

* [Pull] Implementing changes for #802

 - Fix coding style issues with str append
 - Remove useless CACHE_TIMEOUT
 - Use count() instead of $limit
 - Use defaultLinkTo() + handle strings
 - Use http_build_query()
 - Fix missing </em>
 - Remove error_reporting(0)
 - warning CSS (@LogMANOriginal)
 - Fix typo in FeedExpander comment

* [Main] More documentation for markdownToHtml

See #802 for more details
2018-09-09 20:20:13 +01:00
Eugene Molotov
049ee52fb5 Implemented feed item categories (#746) 2018-07-16 12:32:24 +02:00
logmanoriginal
42c699f474 formats: Fix favicon not found if url contains path 2018-06-30 10:27:05 +02:00
teromene
9b34b68180 Do not use an external service in order to fetch the favicon. 2018-05-05 13:55:38 +01:00
logmanoriginal
a4b9611e66 [phpcs] Add missing rules
- Do not add spaces after opening or before closing parenthesis

  // Wrong
  if( !is_null($var) ) {
    ...
  }

  // Right
  if(!is_null($var)) {
    ...
  }

- Add space after closing parenthesis

  // Wrong
  if(true){
    ...
  }

  // Right
  if(true) {
    ...
  }

- Add body into new line
- Close body in new line

  // Wrong
  if(true) { ... }

  // Right
  if(true) {
    ...
  }

Notice: Spaces after keywords are not detected:

  // Wrong (not detected)
  // -> space after 'if' and missing space after 'else'
  if (true) {
    ...
  } else{
    ...
  }

  // Right
  if(true) {
    ...
  } else {
    ...
  }
2017-07-29 19:55:12 +02:00
logmanoriginal
72f40fbd75 [formats] Allow multiple enclosures
All formats now support multiple enclosures. RSS
will show a warning if more than one enclosure
is used since many feed reader don't support
multiple enclosures with RSS (also not clearly
specified in the specification)
2016-11-12 22:04:42 +01:00
logmanoriginal
42cbc2e889 [formats] Fix enclosures
All bridges failed due to missing 'enclosures' element in
the items array. With this commit all formats (ATOM, RSS
and HTML) provide support for a single 'enclosure' element
2016-11-09 18:59:17 +01:00
logmanoriginal
3a2cb9ea1e [formats] Use custom characterset in all formats
The specified characterset will now apply to all formats
thus allowing other charactersets than 'UTF-8'
2016-11-09 18:43:06 +01:00
Marcus Rohrmoser
af572341b3 Fix to PR #427 (#429)
* Ouch, sorry.

* fix indentation.
2016-11-08 21:44:54 +00:00
Marcus Rohrmoser
2ff422d312 Revive enclosures.Reverts 0663c95. Refs #198 #175. 2016-11-08 22:03:51 +01:00
logmanoriginal
ab16af631e [core] Apply some fixes 2016-09-10 21:01:02 +02:00
logmanoriginal
62eec43980 [core] Apply common indentation
All files are now using tabs for indentation
2016-09-10 20:41:11 +02:00
logmanoriginal
e46a480c5d [MrssFormat] Add icon to feed 2016-08-29 20:06:36 +02:00
logmanoriginal
cf146523be [formats] Rename variable 'data' to 'item'
This makes the intend of the variable more clear and is now
coherent with all Bridges
2016-08-29 19:47:21 +02:00
logmanoriginal
a84016bcb6 [core] Rename item getter/setter
getDatas -> getItems
setDatas -> setItems

Note: Bridge->setDatas actually sets data, where Bridge->getItems
only returns items (this is why Bridge->setDatas was not changed)
2016-08-29 19:42:58 +02:00
logmanoriginal
1e9b5c8611 [formats] Remove all unnecessary code comments 2016-08-29 19:24:31 +02:00
logmanoriginal
059b099436 [formats] Fix missing element conditions
This fixes download behavior when requesting Atom/Mrss formats,
which was caused by is_null attempting to expand missing array
elements.
2016-08-29 19:05:41 +02:00
Pierre Mazière
eb3392db82 [core] simplify dynamic formats discovery
Signed-off-by: Pierre Mazière <pierre.maziere@gmx.com>
2016-08-23 14:29:53 +02:00
logmanoriginal
1f3361c6b4 [formats] Change item to associative array
Previously the item class was used to dump literaly any data
into an object. The same can be archived by using an array
with named keys (associative array).

Technically it makes more sense to use an array as we want to
be able to store any parameter into our item.
2016-08-22 18:54:36 +02:00
logmanoriginal
606e756bc1 formats: Use same general entity generation order 2016-08-09 17:12:28 +02:00
logmanoriginal
0d94f1fe0e [MrssFormat] Use XML encoding on data 2016-08-09 17:05:29 +02:00
logmanoriginal
ae716abf65 [MrssFormat] Remove unused and create default URI if non defined 2016-08-09 16:49:27 +02:00
logmanoriginal
be487a6950 [MrssFormat] Link channel to rss-bridge host
This is the same behavior as in AtomFormat
2016-08-09 16:27:32 +02:00
logmanoriginal
f3eefab475 [MrssFormat] Remove media:title and media:content
Both items only make sense in combination with media:content.
This subsequently eradicates $item->thumbnailUri.
2016-08-08 23:28:50 +02:00
ORelio
88bf15e46c Add Author into MrssFormat 2015-09-04 18:33:22 +02:00
nel50n
c8dccf76a6 New Format : Media RSS 2015-03-02 22:35:02 +01:00