rss-bridge/index.php

338 lines
9.1 KiB
PHP
Raw Normal View History

<?php
/*
Create a file named 'DEBUG' for enabling debug mode.
For further security, you may put whitelisted IP addresses in the file,
one IP per line. Empty file allows anyone(!).
Debugging allows displaying PHP error messages and bypasses the cache: this
can allow a malicious client to retrieve data about your server and hammer
a provider throught your rss-bridge instance.
*/
if(file_exists('DEBUG')) {
$debug_whitelist = trim(file_get_contents('DEBUG'));
$debug_enabled = empty($debug_whitelist)
Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802) * Debug mode improvements - Improve debug warning message - Restore error reporting in debug mode - Fix 'notice' messages for unset fields * Add parsing utility functions html.php - extractFromDelimiters - stripWithDelimiters - stripRecursiveHTMLSection - markdownToHtml (partial) bridges - remove now-duplicate functions - call functions from html.php instead * [Anidex] New bridge Anime torrent tracker * [Anime-Ultime] Restore thumbnail * [CNET] Recreate bridge Full rewrite as the previous one was broken * [Dilbert] Minor URI fix Use new self::URI property * [EstCeQuonMetEnProd] Fix content extraction Bridge was broken * [Facebook] Fix "SpSonsSoriSsés" label ... which was taking space in item title * [Futura-Sciences] Use HTTPS, More cleanup Use HTTPS as FS now offer HTTPS Clean additional useless HTML elements * [GBATemp] Multiple fixes - Fix categories: missing "break" statements - Restore thumbnail as enclosure - Fix date extraction - Fix user blog post extraction - Use getSimpleHTMLDOMCached * [JapanExpo] Fix bridge, HTTPS, thumbnails - Fix getSimpleHTMLDOMCached call - Upgrade to HTTPS as JE now offers HTTPS - Restore thumbnails as enclosures * [LeMondeInformatique] Fix bridge, HTTPS - Upgrade to HTTPS as LMI now offers HTTPS - Restore thumbnails using small images - Fix content extraction - Fix text encoding issue * [Nextgov] Fix content extraction - Restore thumbnail and use small image - Field extraction fixes * [NextInpact] Add categories and filtering by type - Offer all RSS feeds - Allow filtering by article type - Implement extraction for brief articles - Remove article limit, many brief articles are publied all at once * [NyaaTorrents] New bridge Anime torrent tracker * [Releases3DS] Cache content, restore thumbnail - Use getSimpleHTMLDOMCached - Restore thumbnail as enclosure * [TheHackerNews] Fix bridge - Fix content extraction including article body - Restore thumbnail as enclosure * [WeLiveSecurity] HTTPS, Fix content extraction - Upgrade to HTTPS as WLS now offers HTTPS - Fix content extraction including article body * [WordPress] Reduce timeout, more content selectors - Reduce timeout to use default one (1h) - Add new content selector (articleBody) - Find thumbnail and set as enclosure - Fix <script> cleanup * [YGGTorrent] Increase limit, use cache - Increase item limit as uploads are very frequent - Use getSimpleHTMLDOMCached * [ZDNet] Rewrite with FeedExpander - Upgrade to HTTPS as ZD now offers HTTPS - Use FeedExpander for secondary fields - Fix content extraction for article body * [Main] Handle MIME type for enclosures Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content). One can force enclosure type using #.ext anchor (hacky, needs improving) * [FeedExpander] Improve field extraction - Add support for passing enclosures - Improve author and uri extraction - Fix 'notice' PHP error messages * [Pull] Coding style fixes for #802 * [Pull] Implementing changes for #802 - Fix coding style issues with str append - Remove useless CACHE_TIMEOUT - Use count() instead of $limit - Use defaultLinkTo() + handle strings - Use http_build_query() - Fix missing </em> - Remove error_reporting(0) - warning CSS (@LogMANOriginal) - Fix typo in FeedExpander comment * [Main] More documentation for markdownToHtml See #802 for more details
2018-09-09 22:20:13 +03:00
|| in_array($_SERVER['REMOTE_ADDR'],
explode("\n", str_replace("\r", '', $debug_whitelist)
)
);
if($debug_enabled) {
ini_set('display_errors', '1');
error_reporting(E_ALL);
define('DEBUG', true);
Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802) * Debug mode improvements - Improve debug warning message - Restore error reporting in debug mode - Fix 'notice' messages for unset fields * Add parsing utility functions html.php - extractFromDelimiters - stripWithDelimiters - stripRecursiveHTMLSection - markdownToHtml (partial) bridges - remove now-duplicate functions - call functions from html.php instead * [Anidex] New bridge Anime torrent tracker * [Anime-Ultime] Restore thumbnail * [CNET] Recreate bridge Full rewrite as the previous one was broken * [Dilbert] Minor URI fix Use new self::URI property * [EstCeQuonMetEnProd] Fix content extraction Bridge was broken * [Facebook] Fix "SpSonsSoriSsés" label ... which was taking space in item title * [Futura-Sciences] Use HTTPS, More cleanup Use HTTPS as FS now offer HTTPS Clean additional useless HTML elements * [GBATemp] Multiple fixes - Fix categories: missing "break" statements - Restore thumbnail as enclosure - Fix date extraction - Fix user blog post extraction - Use getSimpleHTMLDOMCached * [JapanExpo] Fix bridge, HTTPS, thumbnails - Fix getSimpleHTMLDOMCached call - Upgrade to HTTPS as JE now offers HTTPS - Restore thumbnails as enclosures * [LeMondeInformatique] Fix bridge, HTTPS - Upgrade to HTTPS as LMI now offers HTTPS - Restore thumbnails using small images - Fix content extraction - Fix text encoding issue * [Nextgov] Fix content extraction - Restore thumbnail and use small image - Field extraction fixes * [NextInpact] Add categories and filtering by type - Offer all RSS feeds - Allow filtering by article type - Implement extraction for brief articles - Remove article limit, many brief articles are publied all at once * [NyaaTorrents] New bridge Anime torrent tracker * [Releases3DS] Cache content, restore thumbnail - Use getSimpleHTMLDOMCached - Restore thumbnail as enclosure * [TheHackerNews] Fix bridge - Fix content extraction including article body - Restore thumbnail as enclosure * [WeLiveSecurity] HTTPS, Fix content extraction - Upgrade to HTTPS as WLS now offers HTTPS - Fix content extraction including article body * [WordPress] Reduce timeout, more content selectors - Reduce timeout to use default one (1h) - Add new content selector (articleBody) - Find thumbnail and set as enclosure - Fix <script> cleanup * [YGGTorrent] Increase limit, use cache - Increase item limit as uploads are very frequent - Use getSimpleHTMLDOMCached * [ZDNet] Rewrite with FeedExpander - Upgrade to HTTPS as ZD now offers HTTPS - Use FeedExpander for secondary fields - Fix content extraction for article body * [Main] Handle MIME type for enclosures Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content). One can force enclosure type using #.ext anchor (hacky, needs improving) * [FeedExpander] Improve field extraction - Add support for passing enclosures - Improve author and uri extraction - Fix 'notice' PHP error messages * [Pull] Coding style fixes for #802 * [Pull] Implementing changes for #802 - Fix coding style issues with str append - Remove useless CACHE_TIMEOUT - Use count() instead of $limit - Use defaultLinkTo() + handle strings - Use http_build_query() - Fix missing </em> - Remove error_reporting(0) - warning CSS (@LogMANOriginal) - Fix typo in FeedExpander comment * [Main] More documentation for markdownToHtml See #802 for more details
2018-09-09 22:20:13 +03:00
if (empty($debug_whitelist)) {
define('DEBUG_INSECURE', true);
}
}
}
require_once __DIR__ . '/lib/RssBridge.php';
2016-10-08 17:21:00 +03:00
// Specify directory for cached files (using FileCache)
define('CACHE_DIR', __DIR__ . '/cache');
// Specify path for whitelist file
define('WHITELIST_FILE', __DIR__ . '/whitelist.txt');
Configuration::verifyInstallation();
Configuration::loadConfiguration();
Authentication::showPromptIfNeeded();
date_default_timezone_set('UTC');
/*
Move the CLI arguments to the $_GET array, in order to be able to use
rss-bridge from the command line
*/
Catching up | [Main] Debug mode, parse utils, MIME | [Bridges] Add/Improve 20 bridges (#802) * Debug mode improvements - Improve debug warning message - Restore error reporting in debug mode - Fix 'notice' messages for unset fields * Add parsing utility functions html.php - extractFromDelimiters - stripWithDelimiters - stripRecursiveHTMLSection - markdownToHtml (partial) bridges - remove now-duplicate functions - call functions from html.php instead * [Anidex] New bridge Anime torrent tracker * [Anime-Ultime] Restore thumbnail * [CNET] Recreate bridge Full rewrite as the previous one was broken * [Dilbert] Minor URI fix Use new self::URI property * [EstCeQuonMetEnProd] Fix content extraction Bridge was broken * [Facebook] Fix "SpSonsSoriSsés" label ... which was taking space in item title * [Futura-Sciences] Use HTTPS, More cleanup Use HTTPS as FS now offer HTTPS Clean additional useless HTML elements * [GBATemp] Multiple fixes - Fix categories: missing "break" statements - Restore thumbnail as enclosure - Fix date extraction - Fix user blog post extraction - Use getSimpleHTMLDOMCached * [JapanExpo] Fix bridge, HTTPS, thumbnails - Fix getSimpleHTMLDOMCached call - Upgrade to HTTPS as JE now offers HTTPS - Restore thumbnails as enclosures * [LeMondeInformatique] Fix bridge, HTTPS - Upgrade to HTTPS as LMI now offers HTTPS - Restore thumbnails using small images - Fix content extraction - Fix text encoding issue * [Nextgov] Fix content extraction - Restore thumbnail and use small image - Field extraction fixes * [NextInpact] Add categories and filtering by type - Offer all RSS feeds - Allow filtering by article type - Implement extraction for brief articles - Remove article limit, many brief articles are publied all at once * [NyaaTorrents] New bridge Anime torrent tracker * [Releases3DS] Cache content, restore thumbnail - Use getSimpleHTMLDOMCached - Restore thumbnail as enclosure * [TheHackerNews] Fix bridge - Fix content extraction including article body - Restore thumbnail as enclosure * [WeLiveSecurity] HTTPS, Fix content extraction - Upgrade to HTTPS as WLS now offers HTTPS - Fix content extraction including article body * [WordPress] Reduce timeout, more content selectors - Reduce timeout to use default one (1h) - Add new content selector (articleBody) - Find thumbnail and set as enclosure - Fix <script> cleanup * [YGGTorrent] Increase limit, use cache - Increase item limit as uploads are very frequent - Use getSimpleHTMLDOMCached * [ZDNet] Rewrite with FeedExpander - Upgrade to HTTPS as ZD now offers HTTPS - Use FeedExpander for secondary fields - Fix content extraction for article body * [Main] Handle MIME type for enclosures Many feed readers will ignore enclosures (e.g. thumbnails) with no MIME type. This commit adds automatic MIME type detection based on file extension (which may be inaccurate but is the only way without fetching the content). One can force enclosure type using #.ext anchor (hacky, needs improving) * [FeedExpander] Improve field extraction - Add support for passing enclosures - Improve author and uri extraction - Fix 'notice' PHP error messages * [Pull] Coding style fixes for #802 * [Pull] Implementing changes for #802 - Fix coding style issues with str append - Remove useless CACHE_TIMEOUT - Use count() instead of $limit - Use defaultLinkTo() + handle strings - Use http_build_query() - Fix missing </em> - Remove error_reporting(0) - warning CSS (@LogMANOriginal) - Fix typo in FeedExpander comment * [Main] More documentation for markdownToHtml See #802 for more details
2018-09-09 22:20:13 +03:00
if (isset($argv)) {
parse_str(implode('&', array_slice($argv, 1)), $cliArgs);
$params = array_merge($_GET, $cliArgs);
} else {
$params = $_GET;
}
define('REPOSITORY', 'https://github.com/RSS-Bridge/rss-bridge/');
define('USER_AGENT',
'Mozilla/5.0 (X11; Linux x86_64; rv:30.0) Gecko/20121202 Firefox/30.0(rss-bridge/'
. Configuration::$VERSION
. ';+'
. REPOSITORY
. ')'
);
ini_set('user_agent', USER_AGENT);
2014-05-25 21:56:59 +04:00
// default whitelist
$whitelist_default = array(
'BandcampBridge',
'CryptomeBridge',
'DansTonChatBridge',
'DuckDuckGoBridge',
'FacebookBridge',
'FlickrExploreBridge',
'GooglePlusPostBridge',
'GoogleSearchBridge',
'IdenticaBridge',
'InstagramBridge',
'OpenClassroomsBridge',
'PinterestBridge',
'ScmbBridge',
'TwitterBridge',
'WikipediaBridge',
'YoutubeBridge');
2014-05-25 21:56:59 +04:00
2016-09-10 22:01:02 +03:00
try {
Bridge::setDir(__DIR__ . '/bridges/');
Format::setDir(__DIR__ . '/formats/');
Cache::setDir(__DIR__ . '/caches/');
if(!file_exists(WHITELIST_FILE)) {
$whitelist_selection = $whitelist_default;
$whitelist_write = implode("\n", $whitelist_default);
file_put_contents(WHITELIST_FILE, $whitelist_write);
} else {
$whitelist_file_content = file_get_contents(WHITELIST_FILE);
if($whitelist_file_content != "*\n") {
$whitelist_selection = explode("\n", $whitelist_file_content);
} else {
$whitelist_selection = Bridge::listBridges();
}
// Prepare for case-insensitive match
$whitelist_selection = array_map('strtolower', $whitelist_selection);
}
$showInactive = filter_input(INPUT_GET, 'show_inactive', FILTER_VALIDATE_BOOLEAN);
$action = array_key_exists('action', $params) ? $params['action'] : null;
$bridge = array_key_exists('bridge', $params) ? $params['bridge'] : null;
2016-09-10 22:01:02 +03:00
// Return list of bridges as JSON formatted text
if($action === 'list') {
$list = new StdClass();
$list->bridges = array();
$list->total = 0;
foreach(Bridge::listBridges() as $bridgeName) {
$bridge = Bridge::create($bridgeName);
if($bridge === false) { // Broken bridge, show as inactive
$list->bridges[$bridgeName] = array(
'status' => 'inactive'
);
continue;
}
$status = Bridge::isWhitelisted($whitelist_selection, strtolower($bridgeName)) ? 'active' : 'inactive';
$list->bridges[$bridgeName] = array(
'status' => $status,
'uri' => $bridge->getURI(),
'name' => $bridge->getName(),
'icon' => $bridge->getIcon(),
'parameters' => $bridge->getParameters(),
'maintainer' => $bridge->getMaintainer(),
'description' => $bridge->getDescription()
);
}
$list->total = count($list->bridges);
header('Content-Type: application/json');
echo json_encode($list, JSON_PRETTY_PRINT);
} elseif($action === 'display' && !empty($bridge)) {
2016-09-10 22:01:02 +03:00
// DEPRECATED: 'nameBridge' scheme is replaced by 'name' in bridge parameter values
// this is to keep compatibility until futher complete removal
if(($pos = strpos($bridge, 'Bridge')) === (strlen($bridge) - strlen('Bridge'))) {
2016-09-10 22:01:02 +03:00
$bridge = substr($bridge, 0, $pos);
}
$format = $params['format']
or returnClientError('You must specify a format!');
2016-09-10 22:01:02 +03:00
// DEPRECATED: 'nameFormat' scheme is replaced by 'name' in format parameter values
// this is to keep compatibility until futher complete removal
if(($pos = strpos($format, 'Format')) === (strlen($format) - strlen('Format'))) {
2016-09-10 22:01:02 +03:00
$format = substr($format, 0, $pos);
}
// whitelist control
if(!Bridge::isWhitelisted($whitelist_selection, strtolower($bridge))) {
2016-09-10 22:01:02 +03:00
throw new \HttpException('This bridge is not whitelisted', 401);
die;
}
// Data retrieval
$bridge = Bridge::create($bridge);
$noproxy = array_key_exists('_noproxy', $params) && filter_var($params['_noproxy'], FILTER_VALIDATE_BOOLEAN);
if(defined('PROXY_URL') && PROXY_BYBRIDGE && $noproxy) {
2017-02-14 19:28:07 +03:00
define('NOPROXY', true);
}
2016-09-10 22:01:02 +03:00
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
// Cache timeout
$cache_timeout = -1;
if(array_key_exists('_cache_timeout', $params)) {
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
if(!CUSTOM_CACHE_TIMEOUT) {
throw new \HttpException('This server doesn\'t support "_cache_timeout"!');
}
$cache_timeout = filter_var($params['_cache_timeout'], FILTER_VALIDATE_INT);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
} else {
$cache_timeout = $bridge->getCacheTimeout();
}
// Remove parameters that don't concern bridges
$bridge_params = array_diff_key(
$params,
array_fill_keys(
array(
'action',
'bridge',
'format',
'_noproxy',
'_cache_timeout',
'_error_time'
), '')
);
// Remove parameters that don't concern caches
$cache_params = array_diff_key(
$params,
array_fill_keys(
array(
'action',
'format',
'_noproxy',
'_cache_timeout',
'_error_time'
), '')
);
// Initialize cache
$cache = Cache::create('FileCache');
2016-10-08 17:21:00 +03:00
$cache->setPath(CACHE_DIR);
$cache->purgeCache(86400); // 24 hours
$cache->setParameters($cache_params);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
$items = array();
$infos = array();
$mtime = $cache->getTime();
if($mtime !== false
&& (time() - $cache_timeout < $mtime)
&& (!defined('DEBUG') || DEBUG !== true)) { // Load cached data
// Send "Not Modified" response if client supports it
// Implementation based on https://stackoverflow.com/a/10847262
if(isset($_SERVER['HTTP_IF_MODIFIED_SINCE'])) {
$stime = strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']);
if($mtime <= $stime) { // Cached data is older or same
header('Last-Modified: ' . gmdate('D, d M Y H:i:s ', $mtime) . 'GMT', true, 304);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
die();
}
}
$cached = $cache->loadData();
if(isset($cached['items']) && isset($cached['extraInfos'])) {
$items = $cached['items'];
$infos = $cached['extraInfos'];
}
} else { // Collect new data
try {
$bridge->setDatas($bridge_params);
$bridge->collectData();
$items = $bridge->getItems();
$infos = array(
'name' => $bridge->getName(),
'uri' => $bridge->getURI(),
'icon' => $bridge->getIcon()
);
} catch(Error $e) {
error_log($e);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
$item = array();
// Create "new" error message every 24 hours
$params['_error_time'] = urlencode((int)(time() / 86400));
// Error 0 is a special case (i.e. "trying to get property of non-object")
if($e->getCode() === 0) {
$item['title'] = 'Bridge encountered an unexpected situation! (' . $params['_error_time'] . ')';
} else {
$item['title'] = 'Bridge returned error ' . $e->getCode() . '! (' . $params['_error_time'] . ')';
}
$item['uri'] = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) . '?' . http_build_query($params);
$item['timestamp'] = time();
$item['content'] = buildBridgeException($e, $bridge);
$items[] = $item;
} catch(Exception $e) {
error_log($e);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
$item = array();
// Create "new" error message every 24 hours
$params['_error_time'] = urlencode((int)(time() / 86400));
$item['uri'] = parse_url($_SERVER['REQUEST_URI'], PHP_URL_PATH) . '?' . http_build_query($params);
$item['title'] = 'Bridge returned error ' . $e->getCode() . '! (' . $params['_error_time'] . ')';
$item['timestamp'] = time();
$item['content'] = buildBridgeException($e, $bridge);
$items[] = $item;
}
// Store data in cache
$cache->saveData(array(
'items' => $items,
'extraInfos' => $infos
));
}
2016-09-10 22:01:02 +03:00
// Data transformation
try {
$format = Format::create($format);
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
$format->setItems($items);
$format->setExtraInfos($infos);
2018-10-22 20:14:49 +03:00
$format->setLastModified($cache->getTime());
$format->display();
} catch(Error $e) {
error_log($e);
header('Content-Type: text/html', true, $e->getCode());
die(buildTransformException($e, $bridge));
2018-04-06 21:34:44 +03:00
} catch(Exception $e) {
error_log($e);
header('Content-Type: text/html', true, $e->getCode());
Return exceptions in requested feed formats (#841) * [Exceptions] Don't return header for bridge exceptions * [Exceptions] Add link to list in exception message This is an alternative when the button is not rendered for some reason. * [index] Don't return bridge exception for formats * [index] Return feed item for bridge exceptions * [BridgeAbstract] Rename 'getCacheTime' to 'getModifiedTime' * [BridgeAbstract] Move caching to index.php to separate concerns index.php needs more control over caching behavior in order to cache exceptions. This cannot be done in a bridge, as the bridge might be broken, thus preventing caching from working. This also (and more importantly) separates concerns. The bridge should not even care if caching is involved or not. Its purpose is to collect and provide data. Response times should be faster, as more complex bridge functions like 'setDatas' (evaluates all input parameters to predict the current context) and 'collectData' (collects data from sites) can be skipped entirely. Notice: In its current form, index.php takes care of caching. This could, however, be moved into a separate class (i.e. CacheAbstract) in order to make implementation details cache specific. * [index] Add '_error_time' parameter to $item['uri'] This ensures that error messages are recognized by feed readers as new errors after 24 hours. During that time the same item is returned no matter how often the cache is cleared. References https://github.com/RSS-Bridge/rss-bridge/issues/814#issuecomment-420876162 * [index] Include '_error_time' in the title for errors This prevents feed readers from "updating" feeds based on the title * [index] Handle "HTTP_IF_MODIFIED_SINCE" client requests Implementation is based on `BridgeAbstract::dieIfNotModified()`, introduced in 422c125d8e25b9ac5209580b1976fb5d75dab8f8 and simplified based on https://stackoverflow.com/a/10847262 Basically, before returning cached data we check if the client send the "HTTP_IF_MODIFIED_SINCE" header. If the modification time is more recent or equal to the cache time, we reply with "HTTP/1.1 304 Not Modified" (same as before). Otherwise send the cached data. * [index] Don't encode exception message with `htmlspecialchars` * [Exceptions] Include error message in exception * [index] Show different error message for error code 0
2018-10-15 18:21:43 +03:00
die(buildTransformException($e, $bridge));
2018-04-06 21:34:44 +03:00
}
} else {
echo BridgeList::create($whitelist_selection, $showInactive);
2016-09-10 22:01:02 +03:00
}
} catch(HttpException $e) {
error_log($e);
header('Content-Type: text/plain', true, $e->getCode());
2016-09-10 22:01:02 +03:00
die($e->getMessage());
} catch(\Exception $e) {
error_log($e);
2016-09-10 22:01:02 +03:00
die($e->getMessage());
}