mirrors/rss-bridge

Fork 0

mirror of https://github.com/RSS-Bridge/rss-bridge.git synced 2024-11-28 14:18:52 +03:00

Dag 8a6798a227

fix: escape token for html context (#3966 )

2024-02-09 07:27:16 +01:00

15 KiB

Raw Blame History

RSS-Bridge

RSS-Bridge is a PHP web application.

It generates web feeds for websites that don't have one.

Officially hosted instance: https://rss-bridge.org/bridge01/

IRC channel #rssbridge at https://libera.chat/

Full documentation

Alternatively find another public instance.

Requires minimum PHP 7.4.

A subset of bridges (16/447)

CssSelectorBridge: Scrape out a feed using CSS selectors
FeedMergeBridge: Combine multiple feeds into one
FeedReducerBridge: Reduce a noisy feed by some percentage
FilterBridge: Filter a feed by excluding/including items by keyword
GettrBridge: Fetches the latest posts from a GETTR user
MastodonBridge: Fetches statuses from a Mastodon (ActivityPub) instance
RedditBridge: Fetches posts from a user/subredit (with filtering options)
RumbleBridge: Fetches channel/user videos
SoundcloudBridge: Fetches music by username
TelegramBridge: Fetches posts from a public channel
ThePirateBayBridge: Fetches torrents by search/user/category
TikTokBridge: Fetches posts by username
TwitchBridge: Fetches videos from channel
VkBridge: Fetches posts from user/group
XPathBridge: Scrape out a feed using XPath expressions
YoutubeBridge: Fetches videos by username/channel/playlist/search
YouTubeCommunityTabBridge: Fetches posts from a channel's community tab

Tutorial

How to install on traditional shared web hosting

RSS-Bridge can basically be unzipped in a web folder. Should be working instantly.

Latest zip as of Sep 2023: https://github.com/RSS-Bridge/rss-bridge/archive/refs/tags/2023-09-24.zip

How to install on Debian 12 (nginx + php-fpm)

These instructions have been tested on a fresh Debian 12 VM from Digital Ocean (1vcpu-512mb-10gb, 5 USD/month).

timedatectl set-timezone Europe/Oslo

apt install git nginx php8.2-fpm php-mbstring php-simplexml php-curl

# Create a new user account
useradd --shell /bin/bash --create-home rss-bridge

cd /var/www

# Create folder and change ownership
mkdir rss-bridge && chown rss-bridge:rss-bridge rss-bridge/

# Become user
su rss-bridge

# Fetch latest master
git clone https://github.com/RSS-Bridge/rss-bridge.git rss-bridge/
cd rss-bridge

# Copy over the default config
cp -v config.default.ini.php config.ini.php

# Give full permissions only to owner (rss-bridge)
chmod 700 -R ./

# Give read and execute to others (nginx and php-fpm)
chmod o+rx ./ ./static

# Give read to others (nginx)
chmod o+r -R ./static

Nginx config:

# /etc/nginx/sites-enabled/rss-bridge.conf

server {
    listen 80;
    server_name example.com;
    access_log /var/log/nginx/rss-bridge.access.log;
    error_log /var/log/nginx/rss-bridge.error.log;
    log_not_found off;

    # Intentionally not setting a root folder here

    # autoindex is off by default but feels good to explicitly turn off
    autoindex off;

    # Static content only served here
    location /static/ {
        alias /var/www/rss-bridge/static/;
    }

    # Pass off to php-fpm when location is exactly /
    location = / {
        root /var/www/rss-bridge/;
        include snippets/fastcgi-php.conf;
        fastcgi_read_timeout 45s;
        fastcgi_pass unix:/run/php/rss-bridge.sock;
    }

    # Reduce spam
    location = /favicon.ico {
        access_log off;
    }

    # Reduce spam
    location = /robots.txt {
        access_log off;
    }
}

PHP FPM pool config:

; /etc/php/8.2/fpm/pool.d/rss-bridge.conf

[rss-bridge]

user = rss-bridge
group = rss-bridge

listen = /run/php/rss-bridge.sock

listen.owner = www-data
listen.group = www-data

pm = static
pm.max_children = 10
pm.max_requests = 500

PHP ini config:

; /etc/php/8.2/fpm/conf.d/30-rss-bridge.ini

max_execution_time = 15
memory_limit = 64M

Restart fpm and nginx:

# Lint and restart php-fpm
php-fpm8.2 -t
systemctl restart php8.2-fpm

# Lint and restart nginx
nginx -t
systemctl restart nginx

How to install from Composer

Install the latest release.

cd /var/www
composer create-project -v --no-dev rss-bridge/rss-bridge

How to install with Caddy

TODO. See https://github.com/RSS-Bridge/rss-bridge/issues/3785

Install from Docker Hub:

Install by downloading the docker image from Docker Hub:

# Create container
docker create --name=rss-bridge --publish 3000:80 rssbridge/rss-bridge

# Start container
docker start rss-bridge

Browse http://localhost:3000/

Install by locally building from Dockerfile

# Build image from Dockerfile
docker build -t rss-bridge .

# Create container
docker create --name rss-bridge --publish 3000:80 rss-bridge

# Start container
docker start rss-bridge

Browse http://localhost:3000/

Install with docker-compose

Create a docker-compose.yml file locally with with the following content:

version: '2'
services:
  rss-bridge:
    image: rssbridge/rss-bridge:latest
    volumes:
      - </local/custom/path>:/config
    ports:
      - 3000:80
    restart: unless-stopped

Then launch with docker-compose:

docker-compose up

Browse http://localhost:3000/

Other installation methods

The Heroku quick deploy currently does not work. It might work if you fork this repo and modify the repository in scalingo.json. See https://github.com/RSS-Bridge/rss-bridge/issues/2688

Learn more in Installation.

How-to

How to fix "Access denied."

Output is from php-fpm. It is unable to read index.php.

chown rss-bridge:rss-bridge /var/www/rss-bridge/index.php

How to password-protect the instance (token)

Modify config.ini.php:

[authentication]

token = "hunter2"

How to remove all cache items

As current user:

bin/cache-clear

As user rss-bridge:

sudo -u rss-bridge bin/cache-clear

As root:

sudo bin/cache-clear

How to remove all expired cache items

bin/cache-prune

How to fix "PHP Fatal error: Uncaught Exception: The FileCache path is not writable"

# Give rss-bridge ownership
chown rss-bridge:rss-bridge -R /var/www/rss-bridge/cache

# Or, give www-data ownership
chown www-data:www-data -R /var/www/rss-bridge/cache

# Or, give everyone write permission
chmod 777 -R /var/www/rss-bridge/cache

# Or last ditch effort (CAREFUL)
rm -rf /var/www/rss-bridge/cache/ && mkdir /var/www/rss-bridge/cache/

How to fix "attempt to write a readonly database"

The sqlite files (db, wal and shm) are not writeable.

chown -v rss-bridge:rss-bridge cache/*

How to fix "Unable to prepare statement: 1, no such table: storage"

rm cache/*

How to create a new bridge from scratch

Create the new bridge in e.g. bridges/BearBlogBridge.php:

<?php

class BearBlogBridge extends BridgeAbstract
{
    const NAME = 'BearBlog (bearblog.dev)';

    public function collectData()
    {
        $dom = getSimpleHTMLDOM('https://herman.bearblog.dev/blog/');
        foreach ($dom->find('.blog-posts li') as $li) {
            $a = $li->find('a', 0);
            $this->items[] = [
                'title' => $a->plaintext,
                'uri' => 'https://herman.bearblog.dev' . $a->href,
            ];
        }
    }
}

Learn more in bridge api.

How to enable all bridges

enabled_bridges[] = *

How to enable some bridges

enabled_bridges[] = TwitchBridge
enabled_bridges[] = GettrBridge

How to enable debug mode

The debug mode disables the majority of caching operations.

enable_debug_mode = true

How to switch to memcached as cache backend

[cache]

; Cache backend: file (default), sqlite, memcached, null
type = "memcached"

How to switch to sqlite3 as cache backend

type = "sqlite"

How to disable bridge errors (as feed items)

When a bridge fails, RSS-Bridge will produce a feed with a single item describing the error.

This way, feed readers pick it up and you are notified.

If you don't want this behaviour, switch the error output to http:

[error]

; Defines how error messages are returned by RSS-Bridge
;
; "feed" = As part of the feed (default)
; "http" = As HTTP error message
; "none" = No errors are reported
output = "http"

How to accumulate errors before finally reporting it

Modify report_limit so that an error must occur 3 times before it is reported.

; Defines how often an error must occur before it is reported to the user
report_limit = 3

The report count is reset to 0 each day.

How to password-protect the instance (HTTP Basic Auth)

[authentication]

enable = true
username = "alice"
password = "cat"

Will typically require feed readers to be configured with the credentials.

It may also be possible to manually include the credentials in the URL:

https://alice:cat@rss-bridge.org/bridge01/?action=display&bridge=FabriceBellardBridge&format=Html

How to create a new output format

See formats/PlaintextFormat.php for an example.

How to run unit tests and linter

These commands require that you have installed the dev dependencies in composer.json.

./vendor/bin/phpunit
./vendor/bin/phpcs --standard=phpcs.xml --warning-severity=0 --extensions=php -p ./

https://github.com/squizlabs/PHP_CodeSniffer/wiki

How to spawn a minimal development environment

php -S 127.0.0.1:9001

http://127.0.0.1:9001/

Explanation

We are RSS-Bridge community, a group of developers continuing the project initiated by sebsauvage, webmaster of sebsauvage.net, author of Shaarli and ZeroBin.

See CONTRIBUTORS.md

RSS-Bridge uses caching to prevent services from banning your server for repeatedly updating feeds. The specific cache duration can be different between bridges. Cached files are deleted automatically after 24 hours.

RSS-Bridge allows you to take full control over which bridges are displayed to the user. That way you can host your own RSS-Bridge service with your favorite collection of bridges!

Current maintainers (as of 2024): @dvikan and @Mynacol #2519

Reference

Feed item structure

This is the feed item structure that bridges are expected to produce.

    $item = [
        'uri' => 'https://example.com/blog/hello',
        'title' => 'Hello world',
        // Publication date in unix timestamp
        'timestamp' => 1668706254,
        'author' => 'Alice',
        'content' => 'Here be item content',
        'enclosures' => [
            'https://example.com/foo.png',
            'https://example.com/bar.png'
        ],
        'categories' => [
            'news',
            'tech',
        ],
        // Globally unique id
        'uid' => 'e7147580c8747aad',
    ]

Output formats

Atom: Atom feed, for use in feed readers
Html: Simple HTML page
Json: JSON, for consumption by other applications
Mrss: MRSS feed, for use in feed readers
Plaintext: Raw text, for consumption by other applications
Sfeed: Text, TAB separated

Cache backends

File
SQLite
Memcached
Array
Null

Licenses

The source code for RSS-Bridge is Public Domain.

RSS-Bridge uses third party libraries with their own license:

Parsedown licensed under the MIT License
PHP Simple HTML DOM Parser licensed under the MIT License
php-urljoin licensed under the MIT License
Laravel framework licensed under the MIT License

Rant

Dear so-called "social" websites.

Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. FUCK YOU.

You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds.

We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have your service with your applications using your API force-feeding them. Friends must be free to choose whatever software and service they want.

We are rebuilding bridges you have willfully destroyed.

Get your shit together: Put RSS/Atom back in.

15 KiB Raw Blame History

RSS-Bridge

A subset of bridges (16/447)

Tutorial

How to install on traditional shared web hosting

How to install on Debian 12 (nginx + php-fpm)

How to install from Composer

How to install with Caddy

Install from Docker Hub:

Install by locally building from Dockerfile

Install with docker-compose

Other installation methods

How-to

How to fix "Access denied."

How to password-protect the instance (token)

How to remove all cache items

How to remove all expired cache items

How to fix "PHP Fatal error: Uncaught Exception: The FileCache path is not writable"

How to fix "attempt to write a readonly database"

How to fix "Unable to prepare statement: 1, no such table: storage"

How to create a new bridge from scratch

How to enable all bridges

How to enable some bridges

How to enable debug mode

How to switch to memcached as cache backend

How to switch to sqlite3 as cache backend

How to disable bridge errors (as feed items)

How to accumulate errors before finally reporting it

How to password-protect the instance (HTTP Basic Auth)

How to create a new output format

How to run unit tests and linter

How to spawn a minimal development environment

Explanation

Reference

Feed item structure

Output formats

Cache backends

Licenses

Rant

15 KiB

Raw Blame History