The RSS feed for websites missing it
Find a file
Dawid Wróbel b017f75767
[OLXBridge] new bridge (#2944)
* [OLXBridge] new bridge

* [OLXBridge] option to limit to shipping offers only

* [OLXBridge] set the feed title according to search query

* [OLXBridge] Fix PHP notices

* [OLXBridge] Remove trailing slash from the URL

* [OLXBridge] filter out the imposed additional search categories

* [OLXBridge] limit search to 'new' OLX platform variants

* [OLXBridge] Parse date, add ID, description

Deep-crawl all results. Penalty is low, as we were doing this for almost all of the results, anyway, yet it allows to obtain a unique ID, an uncomplicated Date string and a description.

Requires ext-intl for parsing the date according to locale.

* [OLXBridge] Parse date, add ID, description

Deep-crawl all results. Penalty is low, as we were doing this for almost all of the results, anyway, yet it allows to obtain a unique ID, an uncomplicated Date string and a description.

Requires ext-intl for parsing the date according to locale.

* [OLXBridge] Images are optional, handle appropriately

* [OLXBridge] handle the ID coming from sibling auto-moto portal

* [OLXBridge] handle the photos coming from sibling auto-moto portal

* [OLXBridge] use meta property to find img URL

* [OLXBridge] handle the date coming from sibling auto-moto portal

* [OLXBridge] use simplified syntax to retrieve content attribute value

* [OLXBridge] handle the description coming from sibling auto-moto portal

* [OLXBridge] fix phpcs complaints

* [OLXBridge] add categories

* [OLXBridge] handle the categories coming from sibling auto-moto portal

* [OLXBridge] hint image MIME type

OLX images have no obvious extension

* [OLXBridge] Fix content formatting

* [OLXBridge] URL is pattern-checked, so no need to check again

* [OLXBridge] return actual search query as URI
2022-11-17 17:57:05 +01:00
.github fix: prtester css replacement (#3123) 2022-10-29 12:25:45 +02:00
actions feat: add admin telegram contact config (#3134) 2022-11-07 18:36:52 +01:00
bridges [OLXBridge] new bridge (#2944) 2022-11-17 17:57:05 +01:00
cache [TwitterBridge] Fix the bridge using a brand new API 2020-06-05 10:17:53 +02:00
caches fix: make filecache be case-sensitive on key (#3113) 2022-10-29 10:27:26 +02:00
config docker: Switch to nginx in docker image (#2721) 2022-05-11 20:19:25 -04:00
contrib refactor: rename rssbridge.php to bootstrap.php (#2987) 2022-08-27 23:01:06 +02:00
docs docs: Document returnFull parameter of getContents (#3120) 2022-10-29 03:30:49 +02:00
formats refactor: html format (#3062) 2022-10-16 12:03:57 +02:00
lib feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
static feat: system alert message (#3139) 2022-11-15 00:32:04 +01:00
templates feat: system alert message (#3139) 2022-11-15 00:32:04 +01:00
tests refactor: extract frontpage to template (#3130) 2022-11-07 18:22:54 +01:00
vendor Add php-urljoin license 2022-06-09 12:13:07 -04:00
.dockerignore [Docker file] Fix wrong version string in docker images (#2497) 2022-03-14 05:48:40 +05:00
.git-blame-ignore-revs Add .git-blame-ignore-revs file (#2903) 2022-07-08 17:00:34 +02:00
.gitattributes [Docker] force unix line ending on bash files to prevent docker run failing on windows (#2629) 2022-04-10 18:05:43 +02:00
.gitignore docker: Switch to nginx in docker image (#2721) 2022-05-11 20:19:25 -04:00
app.json Update app.json 2022-04-26 22:43:13 +02:00
composer.json [OLXBridge] new bridge (#2944) 2022-11-17 17:57:05 +01:00
composer.lock composer: Update lockfile 2022-07-06 03:34:37 +02:00
config.default.ini.php feat: add max file size to http responses (#3140) 2022-11-16 17:56:26 +01:00
CONTRIBUTORS.md chore: introduce CONTRIBUTORS.md (#2839) 2022-07-04 07:33:23 +02:00
docker-bake.hcl New docker build mechanism (#2268) 2021-10-03 16:06:30 +02:00
docker-entrypoint.sh docker: fix find error (#2733) 2022-05-14 07:18:58 -04:00
Dockerfile [OLXBridge] new bridge (#2944) 2022-11-17 17:57:05 +01:00
index.php refactor: rename rssbridge.php to bootstrap.php (#2987) 2022-08-27 23:01:06 +02:00
phpcompatibility.xml refactor: general code base refactor (#2950) 2022-08-06 22:46:28 +02:00
phpcs.xml refactor: general code base refactor (#2950) 2022-08-06 22:46:28 +02:00
phpunit.xml refactor: rename rssbridge.php to bootstrap.php (#2987) 2022-08-27 23:01:06 +02:00
README.md Update README.md (#3121) 2022-10-29 10:21:51 +02:00
scalingo.json Typo 2016-08-02 23:37:01 +02:00
UNLICENSE let's unlicense ! 2014-05-21 20:11:01 +02:00
whitelist.default.txt core: Move default bridges to whitelist.default.txt 2019-06-06 20:53:46 +02:00

RSS-Bridge

RSS-Bridge

RSS-Bridge is a PHP project capable of generating RSS and Atom feeds for websites that don't have one.

LICENSE GitHub release irc.libera.chat Chat on Matrix Actions Status

Screenshot of the Twitter bridge configuration:

Screenshot #1

Screenshot of the Twitter bridge for Rasmus Lerdorf:

Screenshot #2

Documentation

Check out RSS-Bridge right now on https://rss-bridge.org/bridge01 or find another public instance.

Tutorial

RSS-Bridge requires php 7.4.

Install with git:

cd /var/www
git clone https://github.com/RSS-Bridge/rss-bridge.git

# Give the http user write permission to the cache folder
chown www-data:www-data /var/www/rss-bridge/cache

# Optionally copy over the default config file
cp config.default.ini.php config.ini.php

# Optionally copy over the default whitelist file
cp whitelist.default.txt whitelist.txt

Example config for nginx:

# /etc/nginx/sites-enabled/rssbridge
server {
    listen 80;
    server_name example.com;
    root /var/www/rss-bridge;
    index index.php;

    location ~ \.php$ {
        include snippets/fastcgi-php.conf;
        fastcgi_read_timeout 60s;
        fastcgi_pass unix:/run/php/php-fpm.sock;
    }
}

Install with Docker:

Install by using docker image from Docker Hub:

# Create container
docker create --name=rss-bridge --publish 3000:80 rssbridge/rss-bridge

# Start container
docker start rss-bridge

Browse http://localhost:3000/

Install by locally building the image:

# Build image from Dockerfile
docker build -t rss-bridge .

# Create container
docker create --name rss-bridge --publish 3000:80 rss-bridge

# Start the container
docker start rss-bridge

Browse http://localhost:3000/

Alternative installation methods

Deploy on Scalingo Deploy to Heroku Deploy to Cloudron

The Heroku quick deploy currently does not work. It might possibly work if you fork this repo and modify the repository in scalingo.json. See https://github.com/RSS-Bridge/rss-bridge/issues/2688

Learn more in Installation.

Create a new bridge from scratch

Create the new bridge in e.g. bridges/BearBlogBridge.php:

<?php

class BearBlogBridge extends BridgeAbstract
{
    const NAME = 'BearBlog (bearblog.dev)';

    public function collectData()
    {
        // We can perform css selectors on $dom
        $dom = getSimpleHTMLDOM('https://herman.bearblog.dev/blog/');

        // An array of dom nodes
        $blogPosts = $dom->find('.blog-posts li');

        foreach ($blogPosts as $blogPost) {
            // Select the anchor at index 0 (the first anchor found)
            $a = $blogPost->find('a', 0);

            // Select the inner text of the anchor
            $title = $a->innertext;

            // Select the href attribute of the anchor
            $url = $a->href;

            // Select the <time> tag
            $time = $blogPost->find('time', 0);
            // Create a \DateTime object from the datetime attribute
            $createdAt = date_create_from_format('Y-m-d', $time->datetime);

            $item = [
                'title' => $title,
                'author' => 'Herman',

                // Prepend the url because $url is a relative path
                'uri' => 'https://herman.bearblog.dev' . $url,

                // Grab the unix timestamp
                'timestamp' => $createdAt->getTimestamp(),
            ];

            // Add the item to the list of items
            $this->items[] = $item;
        }
    }
}

Learn more in bridge api.

How-to

How to enable all bridges

Write an asterisks to whitelist.txt:

echo '*' > whitelist.txt

Learn more in enabling briges

How to enable a bridge

Add the bridge name to whitelist.txt:

echo 'FirefoxAddonsBridge' >> whitelist.txt

How to enable debug mode

Create a file named DEBUG:

touch DEBUG

Learn more in debug mode.

How to create a new output format

Create a new format.

Explanation

We are RSS-Bridge community, a group of developers continuing the project initiated by sebsauvage, webmaster of sebsauvage.net, author of Shaarli and ZeroBin.

See CONTRIBUTORS.md

RSS-Bridge uses caching to prevent services from banning your server for repeatedly updating feeds. The specific cache duration can be different between bridges. Cached files are deleted automatically after 24 hours.

RSS-Bridge allows you to take full control over which bridges are displayed to the user. That way you can host your own RSS-Bridge service with your favorite collection of bridges!

Supported output formats:

  • Atom : Atom feed, for use in feed readers
  • Html : Simple HTML page
  • Json : JSON, for consumption by other applications
  • Mrss : MRSS feed, for use in feed readers
  • Plaintext : Raw text, for consumption by other applications

Reference

A selection of bridges

  • Bandcamp : Returns last release from bandcamp for a tag
  • Cryptome : Returns the most recent documents from Cryptome.org
  • DansTonChat: Most recent quotes from danstonchat.com
  • DuckDuckGo: Most recent results from DuckDuckGo.com
  • Facebook : Returns the latest posts on a page or profile on Facebook (There is an issue for public instances)
  • FlickrExplore : Latest interesting images from Flickr
  • GoogleSearch : Most recent results from Google Search
  • Identi.ca : Identica user timeline (Should be compatible with other Pump.io instances)
  • Instagram: Most recent photos from an Instagram user (It is recommended to configure this bridge to work)
  • OpenClassrooms: Lastest tutorials from openclassrooms.com
  • Pinterest: Most recent photos from user or search
  • ScmbBridge: Newest stories from secouchermoinsbete.fr
  • ThePirateBay : Returns the newest indexed torrents from The Pirate Bay with keywords
  • Twitter : Return keyword/hashtag search or user timeline
  • Wikipedia: highlighted articles from Wikipedia in English, German, French or Esperanto
  • YouTube : YouTube user channel, playlist or search

And many more, thanks to the community!

Licenses

The source code for RSS-Bridge is Public Domain.

RSS-Bridge uses third party libraries with their own license:

Rant

Dear so-called "social" websites.

Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. FUCK YOU.

You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds.

We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have your service with your applications using your API force-feeding them. Friends must be free to choose whatever software and service they want.

We are rebuilding bridges you have willfully destroyed.

Get your shit together: Put RSS/Atom back in.