* [FeedMergeBridge] Add new bridge
Here is a bridge that merges two or more feeds into one.
Co-authored-by: Bocki <henning@bocklage.com>
Co-authored-by: Dag <me@dvikan.no>
* New Bridge for Twitter using v2 API
* Top comment block, tweaks to match contributing guide
* [TwitterV2Bridge] new Bridge (sort of)
* Discovered the point of, and re-added, no image scaling option
* Fix the phpcs sniff violations (I hope)
* More linter fixes, I figured out how to use phpcs locally
* Removed unnecessary custom version of getContents function
* Limit query to 100 tweets, valid example query, improved error handling
* Added config doc (correctly, I hope) with link from DESCRIPTION
* little tweak to doc
* Fixup deprecations on PHP 8
Fix#2448
* Configure a default fallback for getInput function
* Appease phpcs
* Avoid changing getInput function
Revert "Configure a default fallback for getInput function"
This reverts commit 94004c5104.
Some timeline items, like review threads and the first comment on PRs,
have no header, so this handles the first comment and adds a generic
title if that doesn't work.
* [CourrierInternationalBridge] fix: skip unusual feed items #2570
This skips feed items who don't have content.
The one I encountered was a horoscope.
This change makes sure the bridge dont errors out.
* [GithubTrendingBridge] Fix bridge: not all languages worked
Languages with more than one word (like "Common Lisp") were not working. Looks like GitHub changed the parameter format: white space is encoded with dashes.
This prompted me to update all languages while I was at it. This also fixed the bug that the C# & F# languages were not working, because the # has to be URL encoded, which is now done in the parameter value. The language "Ren'Py" was commented out. Probably because the single quote was not escaped? I also fixed that.
* [GithubTrendingBridge] Fix PHP notice.
A repo owner can leave the repo description empty, which means the HTML element isn't there. In this case the code produced a PHP notice. This is fixed by checking for null.
* Changed getName() to retrieve the language name directly from the PARAMETERS.
Co-authored-by: dag <me@dvikan.no>
This change instructs browsers to gradually load images
as the user is scrolling down. This is good for performance
because browsers wont download all images right away.
The website keeps activating the Cloudflare protection even on the "non"
standard URL.
To bypass the Cloudflare protection, the bridge tries to resolve a bucnh
of known subdomains, and check if the IP is in a Cloudflare IP range.
If one of the subdomains has a "non Cloudflare" IP, then we use the
CURL_RESOLVE Curl option to connect the unprotected URL using the real
IP instead of the Cloudflare IP.
I hope this will help the bridge to work without needing a fix every week !
There are now two og:url values on the page, the first no longer contains the necessary date URL for the bridge to work. Finding the last og:url on the page restores the bridge to working order.
Sometimes, there are some articles that redirected to another site,
cause the bridge to fail.
Also about disable UID, Reuters frequently updated their article and my
feed reader don't update, I think it's maybe its UID.
First option is session_id of existing Instagram account.
Second option is customizing cache timeout for InstagramBridge.
Those options can be combined.
- increase caching from 2 to 4 hours
- using cURL, extract Last-Modified header of app icons and use as item timestamp
Test warning: F-Droid response time is quite slow even on static assets, the additional requests might impact bridge performance further
This is more like a refactor because they dont serve data in plain
html anymore. Instead, the data is available from a json api
at apibay.org
Could possibly expand this bridge because their api has more to give.
I learned about this api by grokking https://thepiratebay.org/static/main.js
and by looking at browser ajax requests.
For some unknown reason they host some static assets at
https://torrindex.net/ which is used by the bridge to render
magnet image and user status image.
Signed-off-by: Dag <me@dvikan.no>
* Switch ARDMediathekBridge to JSON-API
The html screen scraping approach of ARDMediathekBridge did not work reliably. I could not find one show for which the item list was not empty using the html screen scraping approach.
The proposed change uses the JSON-API of the WebApp. Although there is still room for improvement (feed title, better understanding of the API, more accurate mimic of the webapp's behavior, de-pagination …), it does work with this change.
Indicate that now full URLs as well as just the ID are accepted.
* [PepperBridge] Fix some Notice for expired items
Some expired iteams are not identical to normal items for the title and
URI, so they got a special handling to remove PHP Notice.
The "most commented" sorting option was removed and show now some forum.
So it was removed.
With DealabsBridge, MydealsBridge and HotUKDealsBridge, you can monitor
a discussion for new comments, excluding or not post without URL
* [ExplosmBridge] Rewrite to work without feedburner
re-wrote the bridge to scrap from the new explosm site as the old method of using feedburner is not working anymore, feedburner is stuck on dec/22 when the explosm site changed.
The brand - model page does not have filters now, so the actual feed
using this bridge are broken.
Website has changed. The new website offers a new search function with
many more filter. So I switched to this new search page for this bridge.
CVE Details is a collection of CVEs, taken from the National Vulnerability
Database (NVD) and other sources like the Exploit DB and Metasploit. The
website categorizes it by vendor and product and attach the CWE category.
There is an Atom feed available, but only logged in users can use it,
it is not reliable and contain no useful information. This bridge create a
sane feed with additional information like tags and a link to the CWE
a description of the vulnerability.
The previous value was an int and was not accepted
by rss-bridge as a title. This change uses the image
name instead which fixes the problem and is a better
title than the image id.
* [ReutersBridge] Migrate to new API
- Add new API, feeds.
- Old feed name are perserved for backward compatibility.
- Remove 'Special Report' feed.
- Some feed continue to use old Wire API due to not available in new one.
- Add some new type of content, replace iframe with blockquote for twitter.
- Fix tracking removal in URL
- Fix images broken due to new lazy loading mechanism
- Remove headline, articles do not have it anymore
- Improve article cleanup
New features:
- Add support for custom channel name.
Fixes:
- In playlist mode last uploaded videos were not in feed
- Search mode returned empty feed
And a lot of refactoring
This part of the bridge was meant to find game info on IGN but rarely found useful results, and is harder to maintain than the rest of this bridge due to changes at IGN.
- Add option for case-insensitive regex
- Allow matching item content or author in addition to item title
- Optionally attempt to convert encoding when applying matches
This significantly increases the possibility of missing updates (if
files are uploaded but no file names or post contents are changed) and
of showing an update when there is none (if the post text is changed
but no new files are uploaded). However with the on-page timestamps
removed I'm not sure if there is a good way to do this more accurately
so this is good as we can do at the moment.
- Fix incorrectly working method for querying video info
- Partially fix playlist mode.
At least it works, if playlist has more than 15 videos. But maximum 100 video items are parsed from playlist.
Reason of switching maintainer: https://github.com/RSS-Bridge/rss-bridge/issues/2113#issuecomment-841156902
Sometimes attachments are posted in Telegram channels without any text.
The script recognizes a new message but does not report any text, with this commit the file names will also be included in the RSS Feed.
> The time has come for the Christian Daily Reporter to ride off into the sunset. Disrn is here now and it delivers everything CDR did, times ten.
https://christiandailyreporter.com/
getInstagramSidecarData and getInstagramVideoData were unnecessarily calling getSinglePostData to retrieve data already present in collectData's call of getInstagramJSON. getSinglePostData sometimes doesn't retrieve data properly resulting in incomplete post information. Since the information needed is already present, pass it from collectData instead, eliminating the redundant data collection and improving reliability.
* [TwitchBridge] Switch to unofficial GraphQL API
The GraphQL API that the twitch.tv website uses has a lot more
information available than the official APIs. Hopefully it'll be stable.
Anidex uses two separate domains, anidex.info and anidex.moe
anidex.info has ddos-guard so we need to request
anidex.moe with Host header set to anidex.info
Separate bridge from Releases3DS that just has a different URL.
Inherits from Releases3DS so both bridges need to be present.
* [Releases3DS] Fix PHP notices related to IGN
* [DarkReading] Hide dummy articles
* [FuturaSciences] Strip inline scripts from content
* [FeedExpander] Fix PHP notice on missing uri field
(guid is valid uri AND item uri is not valid)
=> (guid is valid uri AND item uri is empty or not valid)
* [NextInpact] Fix subtitle extraction
* [Markdown] Fix images with empty replacement text
* [TheHackerNews] Fix Author name cleanup
* [LeMondeInformatique] Remove encoding conversion
Was previously needed due to actual encoding on the page
being inconsistent with encoding specified in <meta> tag
* [AnimeUltime] Remove encoding conversion
Was previously needed due to encoding on the page being incorrect
* [FuturaSciences] Fix content extraction
* [FuturaSciences] Fix unneeded unset()
* [GBAtemp] Fix tutorial mode URL extraction
* [GBAtemp] Fix tutorial mode Title extraction
Twitter has changed URL scheme back again (see PR#1647 / commit 78298385d0)
This patch will try both URL schemes now and throw a specific error when neither works
* [DribbbleBridge] Fixed regular expressions for quote replacement in JSON (previously invalid JSON was created if a property value contained colons or single quotes). Also updated two CSS selectors as Dribbble's HTML has changed.
* [DribbbleBridge] Added fix for relative dates in JSON
* [DribbbleBridge] Removed redundant whitespaces
* [TwitterBridge] guest token is returned via body again. This change will try to search fot token inside header and fallback to body
* Twitter changed the URL scheme for the API
* [TwitterBridge] Expire guest token by time
In addition to fetching a new guest token after 100 uses, also expire token after 5 minutes (configurable).
- Remove movies search. It appears the website changed their movies
displays and data cannot be easily extracted for now.
- Fix some errors on items without proper description and/or original
title.
An official RSS feed is available at https://blog.revolut.com/rss/
Note that there is also an invisible "RSS" button next to the Facebook
and Twitter icons at the menu bar.
References #1321
Facebook has changed their strategy regarding permalinks, which
now include lots of unnecessary target data. Fortunately it also
contains the unique story id which we can utilize as URI.
* [InstagramBridge] Fix GraphSidecar output
Fix following issues which related to output of the GraphSidecar type posts.
- The GraphSidecar post's media wasn't outputted except for first picture when searching by hashtag or location
- Video didn't embedded
NOTE:
The function getInstagramStory() which was called when the post type is GraphSidecar didn't seem to work just as one intended.
Because the web request called in that function is just to get the media of single post, NOT to get the media of Story.
But I don't have any idea to solve #694, so it seems be better to rename these function and member variable properly.
This bridge allows you to follow iOS/iPad application updates
directly over RSS. This allows you to get notified of an application
update even if you don't have a iDevice. This may be useful in cases
where you want to be notified of a competitor's application for eg.
I built this after I got tired of waiting for WhatsApp to push their
security release on iOS. It shows up on the AppStore 7 days later
Most of the code in RSS-Bridge uses the long array syntax.
This commit adds a check to enforce using this syntax over
the short array syntax.
All failures have been fixed.
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.
This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
This bridge currently takes a very long time to process
all news items on the page, when in many cases only one
or two had been added since the last check.
This commit adds a new parameter 'limit', which defines
the maximum number of items to add to the feed. This is
an optional paramter that defaults to 3.
* Clarify Facebook bridges status
Distinguish between both Facebook bridges by their title.
This preserves all existing URLs.
* Update all URLs to secure HTTPS versions.
* Configure author name abbreviation
* Improve feed names
Use the correct feed name on each bridge.
Make sure the feed names don't repeat the "Facebook" name.
* [BandcampBridge] Add band and artist feeds
This can return a limited number of the most recent releases by a band,
or a single release/album. Each release may be given a unique article ID
depending on its track list with the "Releases, new one when track track
changes" option, which should make them show up as new articles when
tracks are added or removed. Releases may also be split up to individual
articles for each track with the "Individual tracks" option.
This uses and undocumented API from the Bandcamp Android app. It's much
faster than loading and parsing the website HTML, and seems to fail less
often with more relaxed rate limits. It's still far from perfect in that
regard though.
The "Individual tracks" option generates requests for each individual
track so that can quickly run into rate limits.
The "Individual tracks" option also has a quirk where tracks released
under e.g. a music label will have their artist set to the label instead
of the actual artist of the track. This is a limitation of the API.
* [Rule34pahealBridge] Use HTTPS
* [KonachanBridge] Use HTTPS
* [Rule34Bridge] Use HTTPS
* [SafebooruBridge] Use HTTPS
* [TbibBridge] Use HTTPS
* [XbooruBridge] Use HTTPS
* [ScmbBridge] Use HTTPS
* [ReporterreBridge] Use HTTPS
* [BastaBridge] Use HTTPS
* [NiceMatinBridge] Use HTTPS
* [ScoopItBridge] Use HTTPS
* [TheCodingLoveBridge] Use HTTPS
* [Shimmie2Bridge] Use HTTPS
* [HDWallpapersBridge] Use HTTPS
* [GiphyBridge] Use HTTPS
* [PickyWallpapersBridge] Use HTTPS
* [ParuVenduImmoBridge] Use HTTPS
* [ElsevierBridge] Use HTTPS
* [CastorusBridge] Use HTTPS
* [CollegeDeFranceBridge] Use HTTPS
* [MangareaderBridge] Use HTTPS
* [FacebookBridge] Prevent sending empty header
When running in CLI mode, `getEnv('HTTP_ACCEPT_LANGUAGE')` returns `false`. In that case, don't send the `Accept-Language` header.
* [PatreonBridge] Add new bridge
* [PatreonBridge] Add UID to articles
Patreon changes post URLs when the post title is updated, so set a UID
based on the post ID instead.
The Website changed in two way :
- The filter about availability disappeared (and this leads to a
parameters change, which will break existing bridges, sorry)
- Some HTML change
Directly accessing ...->href resulted in a string that contained '&'
instead of '&'. This was later escaped again to '&amp;' in some
formats (e.g. Atom).
Removes 'self::URI' from processUpload() which was creating malformed URLs. Relative URLs are handled by defaultLinkTo() making 'self::URI' unnecessary.
* [TwitterBridge] Get cookies before sending request
Twitter now requires cookies to be set before requesting a page. This will fetch the cookies and send them to `getSimpleHTMLDOM()`.
* Formatting fixes
The public service demonoid.pw is no longer available and is
currently being rebuild under demonoid.info which hides torrents
behind a login wall. As this is not supported by RSS-Bridge, the
bridge will be removed.
Find more details on Reddit:
https://www.reddit.com/r/Demonoid/
* Remove getSearchResults()
* Remove ''From search' from PARAMETERS array
* Update getURI() and getName()
* Update collectData()
* Add '.rss' to URL in `collectData` instead of in `getURI`
Implements a bridge for
https://www.indeed.com/ (or any of the local variants)
Features:
- Takes a company name and returns a list of reviews and comments
- Limit the maximum number of items to return (default: 20)
- No upper limit on the number of items to return
- Search by language code (45 options)
- Supports detectParameters for any supported URL
The cache factory can be based on the abstract factory class if it
wasn't static. This allows for higher abstraction and makes future
extensions possible. Also, not all parts of RSS-Bridge need to work
on the same instance of the factory.
References #1001
detectParameters() is called in a loop for all bridges on a URL, thus
if a bridge returns an error message, the output messages get mixed
up and all detect operations fail.
This seems to be a limitation of the detect function for now.
The target site for this bridge has been down for at least a year
now and there doesn't seem to be any attempt to get it back up.
Their twitter account is also silent since 2012, so no harm
removing this bridge.
https://twitter.com/dollbooru?lang=en
- Add function to build an URL to the GitHub issue comment
- Change scope of internal functions from protected to private
- Use IDs instead of classes as comment selectors, to include the
issue author in the output feed.
References #1100
This reverts commit 052844f5e1.
There is a bug in ->remove() that causes the parser to incorrectly
identify elements in the DOM tree that shouldn't exist anymore.
References #1151
simplehtmldom 1.9 introduced new functions to recursively remove
nodes from the DOM. This allows removing elements without the need
to re-load the document by using $html->load($html->save()), which
is very inefficient.
Find more information about remove() at
https://simplehtmldom.sourceforge.io/docs/1.9/api/simple_html_dom_node/remove/
- For consistency, functions should always return null on non-existing data.
- WordPressPluginUpdateBridge appears to have used its own cache instance in the past. Obviously not used anymore.
- Since $key can be anything, the cache implementation must ensure to assign the related data reliably; most commonly by serializing and hashing the key in an appropriate way.
- Even though the default path for storage is perfectly fine, some people may want to use a different location. This is an example how a cache implementation is responsible for its requirements.
* [BingSearch] Make the bridge compatible with PHP 5.6
The use of isset() with an expression is not possible in PHP 5.6. I
fixed it by replacing isset() with "null !== ".
This seems like a weird bug somewhere.
Either the HTML parser should return the valid page, or the CMS should
not convert the URL first, or the URL validation regex is buggy.
* [AutoJMBridge] Fix bridge after website change
The website was totally reworked, so the bridge had to be reworked too.
The bridge parameters changed, therefore old RSS feed will not work
anymore, but it was impossible to do it in another way.
In case of a unparsable date, the text to DateTime object failed, and
this resulted to a Fatal error while using this DateTime object . To
prvent this fatal error, if the date parsing failse, then a DateTime
object is created with the actual date.
* rework to parse new layout
* skip incomplete rows
The last row could have fewer columns if there are less rows than the items limit. This usually should not happen, though.
* use constant for skipping
Bandcamp tags pages have a new layout and now use a POST API endpoint to view each page of releases.
Output of this bridge should be almost the same as before, with a few small improvements:
- Small album image in 'content', larger album image in 'enclosures'
- RSS item titles/authors are appended with the releaser in parentheses if the artist name and the releaser are different (i.e. Record Label's Bandcamp releases an album called Bar by the band named Foo, it would get the title 'Foo - Bar (Record Label)' and the author 'Foo (Record Label)')
* Fix issue with CachetAPI Pagination
Fixing issue that only the oldest 20 entries were shown.
_Background:_
_Cachet has a, lets call it odd, system of pagination. On the first page you see the incidents first created, so they are not what you want to see. But on the last page you can have 1 or 20 of the newest incidents. So you have to take the incidents from the last page (call it Pmax) and combine them with the incidents from Pmax - 1._
This commit adds support for a new parameter which specifies the type
of cache to use for caching. It is specified in config.ini.php:
[cache]
type = "..."
Currently only one type of cache is supported (see /caches). All uses
of 'FileCache' were replaced by this configuration option.
Note: Caching currently depends on files and folders (due to FileCache).
Experience may vary depending on the selected cache type. For now always
check if FileCache is working before testing alternative types.
References #1000
In item content, the image is now a clickable link to the post;
In item title a ▶ is prepended if the post contains a video; it's impossible to tell from the content image.
Images in Twitter feeds are currently being scaled by adding ':orig'
(original image) and ':thumb' (thumbnail) to image URIs in the feed.
This can cause issues with feed readers that don't handle colons in
URIs correctly.
Image scaling can now be disabled by adding '&noimgscaling=on' to the
query. This parameter is optional to stay compatible to existing feeds.
References #957
1. Author Name now doesn't include Episode Title
2. It now fetches Episode Creation Timestamp, to allow correct sorting in podcatchers
3. Description is now the actual show notes, and not an <audio> tag
* core: Add bridge parameter auto detection
This adds a new 'detect' action which accepts a URL from which an
appropriate bridge is selected and relevant parameters are extracted.
The user is then automatically redirected to the selected bridge.
For example to get a feed from: https://twitter.com/search?q=%23rss-bridge
we could send a request to:
'/?action=detect&format=Atom&url=twitter.com/search%3Fq%3D%2523rss-bridge'
which would redirect to:
'/?action=display&q=%23rss-bridge&bridge=Twitter&format=Atom'.
This auto detection happens on a per-bridge basis, so a new function
'detectParameters' is added to BridgeInterface which bridges may implement.
It takes a URL for an argument and returns a list of parameters that were
extracted, or null if the URL isn't relevant for the bridge.
* [TwitterBridge] Add parameter auto detection
* [BridgeAbstract] Add generic parameter detection
This adds generic "paramater detection" for bridges that don't have any
parameters defined. If the queried URL matches the URI defined in the
bridge (ignoring https://, www. and trailing /) an emtpy list of parameters is
returned.
* [ZoneTelechargementBridge] Switch to the new Website
The website zone-telechargement1.org decided that he will be using a new
domain at the end of november :
https://www.annuaire-telechargement.com/
The bridge uses the new domain but still uses the same filename and
class name to keep the existing feed working.
Replaces 'debugMessage' by specialized debug function 'Debug::log'.
This function takes the same arguments as the previous 'debugMessage'.
A separate Debug class allows for further optimization and separation
of concern.
This commit adds filters to remove embedded videos and view counts from
all posts. This doesn't remove the preview image for videos, which are
embedded separately.
Hidden elements are used for error conditions and generally made
visible using JavaScript. Since RSS-Bridge doesn't support JS, these
error messages are shown in the final feed. For example:
"It looks like you may be having problems playing this video. If so,
please try restarting your browser."
This commit removes all hidden elements to prevent error messages being
added to the feed.
- "It looks like you may be having problems playing this video. If so,
please try restarting your browser."
FB includes origin information (i.e. "YOUTUBE.COM") as well as
descriptions with embedded media (images and video).
These details are currently being removed by the bridge.
This commit changes implementation to only remove origin information
and keep the media description in place. The media description consists
of two elements - title and description. The title provided by FB is
included in an anchor, which gets replaced by a paragraph with the
same contents to improve readability.
References #912
Move CACHE_DIR from index.php to /lib/RssBridge.php and change name
to PATH_CACHE.
PATH_CACHE is one of the core paths of RSS-Bridge and should therefore
be defined in the core file RssBridge.php.
`defaultLinkTo` makes anchors point to the correct path which broke
parsing because it expected href to start with `#gistcomment`.
This commit changes the implementation to make `defaultLinkTo` point
to the correct page (using `getURI` instead of `self::URI`) and search
with `*=` instead of `^=`.
Instagram returns "HTTP/1.1 301 Moved Permanently" on each request
to "https://instagram.com/" because the correct location is
"https://www.instagram.com/".
Instagram will respond with "HTTP/1.1 301 Moved Permanently" if the
URI for the requested user doesn't end with a slash.
Notice: This is only a minor enhancement to prevent error 301 from
happening. The previous version worked fine as is.
Using the classical www.zone-telechargement1.org as base URL, the bridge will
always be redirected to the actual wwX.zone-telechargement1.org final URL. This
makes the bridge more robust to URL changes.
This commit collects the original contents from a different
tag to prevent this issue. The root cause is unknown but closely
related to the regex.
References #877
The function 'defaultLinkTo' applied to the source HTML does break
regex matches later in the bridge. We need to apply the function
right before adding the contents to the item for the bridge to work
properly.
References #856
- Base URL updated
- Show name has different styles on the Website, use another way to get the show name
- Entry URIs are now unique to make sure RSS readers don't treat episodes as duplicates
- No more new lines in the feed or item title
- Entry URIs are unique to make sure RSS readers don't treat episodes as duplicates
- Titles are unique to make sure RSS readers don't treat streams and downloads as duplicates
The bridge has been expanded to better cover the whole GQ magazine.
It should support all countries (provided they all use the same absurdly shitty publication system).
It is guaranteed to be only tested with sexactu articles (that I now obtain by loading Maïa Mazaurette author page).
This commit adds a new optional parameter 'limit' which can be used
to limit the number of items returned by this bridge (i.e. '&limit=10')
As requested in #669
The URI "https://facebook.com/username?_fb_noscript=1" returns two
posts per user. Some profiles, however, are very active, causing the
bridge to miss items if more than two posts are send within the cache
duration (5 minutes).
The alternative suggested in #669 is to use a different URI:
"https://facebook.com/pg/username/posts?_fb_noscript=1"
While the contents of this URI essentially look the same when viewed
in a browser, it actually returns more than 10 posts depending on the
profile.
References #669
* Fix typo with status field
* Comply with other bridges
Change the uri element of an item to point, not on the magnet link, but on the page, as similar bridges do.
* Improved to return name & uri matching with query
This change makes it possible for the feed reader to discover a title and url consistent with the user's search.
* [ZoneTelechargementBridge] Added Bridge for ww2.zone-telechargement1.org
Goal for this bridge is to follow the episode publication of a TV show
season while it's broadcasted on the TV.
* [AutoJMBridge] New Bridge
This bridge will show all the car offers AutoJM has for the model you
choosed and using your filter. Very useful to wait for a cheap price for
a new car !
This commit fixes#782 by updating the parameter value of 'Maison &
Jardin', but this means the user has to update his RSS Feed URL (.because
of the bridge structure, it would be a nightmare to fix it in another
way)
This commits add all the categories available on Dealabs Website.
* [ExtremeDownloadBridge] Added Bridge for ww1.extreme-d0wn.com
Goal for this bridge is to follow the episode publication of a TV show season
while it's broadcasted on the TV.
* Adding a bridge for theyetee.com
The bridge fetches daily shirts at theyetee.com.
The Yetee offers two new shirts each day, but you can buy them only for a few hours !
Unfortunately, the site don't provide RSS feed, so the only way to keep up to date on new shirt is their daily mailing ... until now !
* Debug mode improvements
- Improve debug warning message
- Restore error reporting in debug mode
- Fix 'notice' messages for unset fields
* Add parsing utility functions
html.php
- extractFromDelimiters
- stripWithDelimiters
- stripRecursiveHTMLSection
- markdownToHtml (partial)
bridges
- remove now-duplicate functions
- call functions from html.php instead
* [Anidex] New bridge
Anime torrent tracker
* [Anime-Ultime] Restore thumbnail
* [CNET] Recreate bridge
Full rewrite as the previous one was broken
* [Dilbert] Minor URI fix
Use new self::URI property
* [EstCeQuonMetEnProd] Fix content extraction
Bridge was broken
* [Facebook] Fix "SpSonsSoriSsés" label
... which was taking space in item title
* [Futura-Sciences] Use HTTPS, More cleanup
Use HTTPS as FS now offer HTTPS
Clean additional useless HTML elements
* [GBATemp] Multiple fixes
- Fix categories: missing "break" statements
- Restore thumbnail as enclosure
- Fix date extraction
- Fix user blog post extraction
- Use getSimpleHTMLDOMCached
* [JapanExpo] Fix bridge, HTTPS, thumbnails
- Fix getSimpleHTMLDOMCached call
- Upgrade to HTTPS as JE now offers HTTPS
- Restore thumbnails as enclosures
* [LeMondeInformatique] Fix bridge, HTTPS
- Upgrade to HTTPS as LMI now offers HTTPS
- Restore thumbnails using small images
- Fix content extraction
- Fix text encoding issue
* [Nextgov] Fix content extraction
- Restore thumbnail and use small image
- Field extraction fixes
* [NextInpact] Add categories and filtering by type
- Offer all RSS feeds
- Allow filtering by article type
- Implement extraction for brief articles
- Remove article limit, many brief articles are publied all at once
* [NyaaTorrents] New bridge
Anime torrent tracker
* [Releases3DS] Cache content, restore thumbnail
- Use getSimpleHTMLDOMCached
- Restore thumbnail as enclosure
* [TheHackerNews] Fix bridge
- Fix content extraction including article body
- Restore thumbnail as enclosure
* [WeLiveSecurity] HTTPS, Fix content extraction
- Upgrade to HTTPS as WLS now offers HTTPS
- Fix content extraction including article body
* [WordPress] Reduce timeout, more content selectors
- Reduce timeout to use default one (1h)
- Add new content selector (articleBody)
- Find thumbnail and set as enclosure
- Fix <script> cleanup
* [YGGTorrent] Increase limit, use cache
- Increase item limit as uploads are very frequent
- Use getSimpleHTMLDOMCached
* [ZDNet] Rewrite with FeedExpander
- Upgrade to HTTPS as ZD now offers HTTPS
- Use FeedExpander for secondary fields
- Fix content extraction for article body
* [Main] Handle MIME type for enclosures
Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content).
One can force enclosure type using #.ext anchor (hacky, needs improving)
* [FeedExpander] Improve field extraction
- Add support for passing enclosures
- Improve author and uri extraction
- Fix 'notice' PHP error messages
* [Pull] Coding style fixes for #802
* [Pull] Implementing changes for #802
- Fix coding style issues with str append
- Remove useless CACHE_TIMEOUT
- Use count() instead of $limit
- Use defaultLinkTo() + handle strings
- Use http_build_query()
- Fix missing </em>
- Remove error_reporting(0)
- warning CSS (@LogMANOriginal)
- Fix typo in FeedExpander comment
* [Main] More documentation for markdownToHtml
See #802 for more details
The bridge would generate empty titles if the content is longer than
50 characters, but doesn't have further spaces in it. With this commit
the title is correctly generated based on the contents, taking missing
spaces into account.
References #786
https://cad-comic.com/ now provides feeds at
- https://cad-comic.com/feed (rss)
- https://cad-comic.com/feed/atom (atom)
Thus multiple alternatives are available to choose from, making this
bridge obsolete:
- FilterBridge (using one of the feeds above)
- WordPressBridge (on the main site)
- One of the two available feeds
References #752
This commit fixes an issue caused by self closing tags not supported
by simplehtmldom (<source>).
Adds a monkey patch to extend simplehtmldom with the ability to detect
that particular tag. Most of the code added is copied directly from
simplehtmldom (see vendor/simplehtmldom) with adjustments to account
for RSS-Bridge formatting.
Related to: https://sourceforge.net/p/simplehtmldom/bugs/83/
Notice: The tag itself is valid according to Mozilla:
The HTML <picture> element serves as a container for zero or more
<source> elements and one <img> element to provide versions of an
image for different display device scenarios. The browser will
consider each of the child <source> elements and select one
corresponding to the best match found; if no matches are found
among the <source> elements, the file specified by the <img>
element's src attribute is selected. The selected image is then
presented in the space occupied by the <img> element.
-- https://developer.mozilla.org/en-US/docs/Web/HTML/Element/picture
References #753
* use defaultLinkTo
* remove duplicate video links
* remove line ending before "Reposted" label
* return newline before reposted string
* remove comments
* use video links that won't require login
* set title if video has no title
Adds a new option '&title_from_content=on' to build the title for feed
items from the feeds content. The title is generated from the first
whitespace after 50 characters of the content or the entire content if
the total size is lower than 50 characters.
References #587
