owncast/doc/S3.md
2020-06-10 23:59:43 -07:00

7 KiB

Here are some setup details, general information and tips specific to using external storage.

Choose your storage provider of choice. Yours not listed? File an issue and we'll test and write up some documentation for it.

Linode Object Storage

250 GB storage + 1 TB Outbound Transfer for $5/mo.

Linode Object Storage is a good choice if you're already using Linode to host your server. It should be fast to transfer your video from your server to their storage service, and their pricing will probably just be the flat $5/mo for you, so it's easy to know what you're paying.

  • Create a new bucket at the Linode Object Storage admin page.
  • Edit your config file and change the S3 endpoint to match the hostname listed below your newly created bucket that looks something like myvideo.us-east-1.linodeobjects.com, the bucket name to match the one you just created and the S3 region to match the us-east-1 equivalent of the above hostname.
  • Using the Linode Object Access Keys page create a new Access Key and add the Key and Secret to your config.yaml file.

In the following steps Linode requires you to interact with your bucket using the s3cmd tool. So install that on your terminal and configure it.

Run s3cmd --configure and fill in the values with what is currently in your config file. It should look similar to this:

Access Key: ABC12334
Secret Key: fj3kd83jdkh
Default Region: US
S3 Endpoint: us-east-1.linodeobjects.com
DNS-style bucket+hostname:port template for accessing a bucket: us-east-1.linodeobjects.com
Use HTTPS protocol: False

Add permissions to access video.

This part sucks. But you only have to do it once per bucket. These are the full instructions but let me summarize.

  1. Create a file called bucket_policy.json that has the following:
{
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "*"
        ]
      },
      "Action": [
        "s3:GetObject"
      ],
      "Resource": [
        "arn:aws:s3::MYBUCKETNAME/*"
      ]
    }
  ]
}
  1. Replace MYBUCKETNAME with your actual bucket name.
  2. Run s3cmd setpolicy bucket_policy.json s3://MYBUCKETNAME replacing MYBUCKETNAME with your bucket name.
  3. Run s3cmd info s3://MYBUCKETNAME to make sure the new policy saved.

Now files video saved to Linode Object Storage will be readable.

More details about how to get started using Linode Object Storage can be found on their documentation.

File expiration

Make files older than one day expire and delete themselves so you don't pay for storage of old video.

Full details are in their documentation.

Create a file called lifecycle_policy.json with the following contents:

<LifecycleConfiguration>
    <Rule>
        <ID>delete-all-objects</ID>
        <Prefix></Prefix>
        <Status>Enabled</Status>
        <Expiration>
            <Days>1</Days>
        </Expiration>
    </Rule>
</LifecycleConfiguration>
  • Run s3cmd setlifecycle lifecycle_policy.xml s3://MYBUCKETNAME.
  • Run s3cmd info s3://MYBUCKETNAME and you should now see Expiration Rule: all objects in this bucket will expire in '1' day(s) after creation.

AWS S3

AWS S3 is a good choice if you're already using AWS for your server or are comfortable using AWS for other things. If you're brand new to object storage and not using AWS already I'm not sure I'd recommend jumping into it just for Owncast. There are other options.

Here's some example usage and pricing for AWS S3:

File expiration

You should expire old segments on your S3 bucket. Here are some instructions on how to do that.

  • Once an object expires you won't be charged for storage, even if the object isn't deleted immediately.
  • A one day object expiration lifecycle rule on objects is as low as you can go, so use that.
  • Because AWS rounds the expiration to midnight of the next day you may have a lot of old video chunks sitting around. You can make the most of this by increasing the maxNumberInPlaylist value in your config file to something much higher, allowing users to rewind your stream back in time further. If the video is available then you might as well make it available to your users.

CORS

Wasabi cloud storage

Most people would end up paying $5.99/mo with Wasabi, and there's no additional costs for the amount of people or amount of times people access your video. So if you have a ton of viewers, this is probably a good option.

Create a user and access key.

  1. Create a new user on Wasabi for yourself.
  2. Create a new Access Key in the Users Panel.
  3. Update your Owncast config.json file with the above Access Key and Secret as well as the other required details.

Depending on the region where your bucket lives look up the service URL from this page.

  • Endpoint: Is the "service URL" you looked up above. Likely s3.wasabisys.com or similar.
  • Bucket

Making files public

Wasabi makes it easy to make a bucket public. Full documentation is here, but simply select the folder and choose "Make Public".

Expiration of old files on Wasabi

Important note! Wasabi does *NOT seem to have a way to set a policy for deleting old files like AWS and Linode does. You may have your own way of cleaning up old files, or some other solution. But it's something to keep in mind in case you really start to build up a lot of old video files.

If anybody knows how to enable Lifecycle Policies on Wasabi, please file an issue with details.