owncast/core/data/config.go
Gabe Kangas bc2caadb74
0.0.6 -> Master (#731)
* Implement webhook events for external integrations (#574)

* Implement webhook events for external integrations

Reference #556

* move message type to models and remove duplicate

* add json header so content type can be determined

* Pass at migrating webhooks to datastore + management apis (#589)

* Pass at migrating webhooks to datastore + management apis

* Support nil lastUsed timestamps and return back the new webhook on create

* Cleanup from review feedback

* Simplify a bit

Co-authored-by: Aaron Ogle <aaron@geekgonecrazy.com>

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Webhook query cleanup

* Access tokens + Send system message external API (#585)

* New add, get and delete access token APIs

* Create auth token middleware

* Update last_used timestamp when using an access token

* Add auth'ed endpoint for sending system messages

* Cleanup

* Update api spec for new apis

* Commit updated API documentation

* Add auth'ed endpoint for sending user chat messages

* Return access token string

* Commit updated API documentation

* Fix route

* Support nil lastUsed time

* Commit updated Javascript packages

* Remove duplicate function post rebase

* Fix msg id generation

* Update controllers/admin/chat.go

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Webhook query cleanup

* Add SystemMessageSent to EventType

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Set webhook as used on completion. Closes #610

* Display webhook errors as errors

* Commit updated API documentation

* Add user joined chat event

* Change integration API paths. Update API spec

* Update development version of admin that supports integration apis

* Commit updated API documentation

* Add automated tests for external integration APIs

* check error

* quiet this test for now

* Route up some additional 3rd party apis. #638

* Commit updated API documentation

* Save username on user joined event

* Add missing scope to valid scopes list

* Add generic chat action event API for 3rd parties. Closes #666

* Commit updated API documentation

* First pass at moving WIP config framework into project for #234

* Only support exported fields in custom types

* Using YP get/set key as a first pass at using the data layer. Fixes + integration.

* Ignore test db

* Start adding getters and setters for config values

* More get/set config work. Starting to populate api with data

* Wire up some config edit endpoints

* More endpoints

* Disable cors middleware

* Add more endpoints and add test to test them

* Remove the in-memory change APIs

* Add endpoint for changing tags

* Add more config endpoints

* Starting to point more things away from config file and to the datastore

* Populate YP with db data

* Create new util method for parsing page body markdown and return it in api

* Verify proposed path to ffmpeg

* For development purposes show the config key in logs

* Move stats values to datastore

* Moving over more values to the datastore

* Move S3 config to datastore

* First pass the config -> db migrator

* Add the start of the video config apis

* It builds pointing everything away from the config

* Tweak ffmpeg path error message

* Backup database every hour. Closes #549

* Config + defaults + migration work for db

* Cleanup logging

* Remove all the old config structs

* Add descriptive info about migration

* Tweak ffmpeg validation logic

* Fix db backup path. backup on db version migration

* Set video and s3 configurations

* Update api spec with new config endpoints

* Add migrator for stats file

* Commit updated API documentation

* Use a dynamic system port for internal HLS writes. Closes #577 (#626)

* Use a dynamic system port for internal HLS writes. Closes #577

* Cleanup

* YP key migration to datastore

* Create a backup directory if needed before migrations

* Remove config test that no longer makes sense. Cleanup.

* Change number types from float32 to float64

* Update automated test suite

* Allow restoring a database backup via command line flags. Closes #549

* Add new hls segment config api

* Commit updated API documentation

* Update apis to require a value container property

* add socialHandles api

* Commit updated API documentation

* Add new latancy level setting to replace segment settings

* Commit updated API documentation

* Fix spelling

* Commit updated API documentation

* hardcode a json api of available social platforms

* Add additional icons

* Return social handles in server config api

* Add socialhandles validation to test

* Move list of hard coded social platforms to an api

* Remove audio only code from transcoder since we do not use it

* Add latency levels api + snapshot of video settings as current broadcast

* Add config/serverurl endpoint

* Return 404 on YP api if disabled

* Surface stream title in YP response

* Add stream title to web ui

* Cleanup log message. Closes #520

* Rename ffmpeg package to transcoder

* Add ws package for testing

* Reduce chat backlog to past 5hrs, max 50. Closes #548

* Fix error formatting

* Add endpoint for resetting yp registration

* Add yp/reset to api spec. return status in response

* Return zero viewer count if stream is offline. Closes #422

* Post-rebase fixes

* Fix merge conflict in openapi file

* Commit updated API documentation

* Standardize controller names

* Support setting the stream key via the command line. Closes #665

* Return social handles with YP data. First half of https://github.com/owncast/owncast-yp/issues/28

* Give the YP package access to server status regardless if enabled or not

* Change delay in automated tests

* Add stream title integration API. For #638

* Commit updated API documentation

* Add storage to the migrator

* Missing returning NSFW value in server config

* Add flag to ignore websocket client. Closes #537

* Add error for parsing broadcaster metadata

* Add support for a cli specified http server port. Closes #674

* Add cpu usage levels and a temporary mapping between it and libx264 presets

* Test for valid url endpoint when saving s3 config

* Re-configure storage on every stream to allow changing storage providers

* After 5 minutes of a stream being stopped clear the stream title

* Hide viewer count once stream goes offline instead of when player stops

* Pull steamTitle from the status that gets updated instead of the config

* Commit updated API documentation

* Optionally show stream title in the header

* Reset stream title when server starts

* Show chat action when stream title is updated

* Allow system messages to come back in persistence

* Split out getting chat history for moderation + fix tests

* Remove server title and standardize on name only

* Commit updated API documentation

* Bump github.com/aws/aws-sdk-go from 1.37.1 to 1.37.2 (#680)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.1 to 1.37.2.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.1...v1.37.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add video variant and stream latency config file migrator

* Remove mostly unused disable upgrade check bool

* Commit updated API documentation

* Allow bundling the admin from the 0.0.6 branch

* Fix saving port numbers

* Use name instead of old title on window focus

* Work on latency levels. Fix test to use levels. Clean up transcoder to only reference levels

* Another place where title -> name

* Fix test

* Bump github.com/aws/aws-sdk-go from 1.37.2 to 1.37.3 (#690)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.2 to 1.37.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.2...v1.37.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update dependabot config

* Bump github.com/aws/aws-sdk-go from 1.37.3 to 1.37.5 (#693)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.3 to 1.37.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.3...v1.37.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript (#694)

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript

Bumps [video.js](https://github.com/videojs/video.js) from 7.10.2 to 7.11.4.
- [Release notes](https://github.com/videojs/video.js/releases)
- [Changelog](https://github.com/videojs/video.js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/videojs/video.js/compare/v7.10.2...v7.11.4)

Signed-off-by: dependabot[bot] <support@github.com>

* Commit updated Javascript packages

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>

* Make the latency migrator dynamic so I can tweak values easier

* Split out fetching ffmpeg path from validating the path so it can be changed in the admin

* Some commenting and linter cleanup

* Validate the path for a logo change and throw an error if it does not exist

* Logo change requests have to be a real file now

* Cleanup, making linter happy

* Format javascript on push

* Only format js in master

* Tweak latency level values

* Remove unused config file examples

* Fix thumbnail generation after messing with the ffmpeg path getter

* Reduce how often we report high hardware utilization warnings

* Bundle the 0.0.6 branch version of the admin

* Return validated ffmpeg path in admin server config

* Change the logo to be stored in the data directory instead of webroot

* Bump postcss from 8.2.4 to 8.2.5 in /build/javascript (#702)

Bumps [postcss](https://github.com/postcss/postcss) from 8.2.4 to 8.2.5.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.2.4...8.2.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Default config file no longer used

* don't show stream title when offline

addresses https://github.com/owncast/owncast/issues/677

* Remove auto-clearing stream title. #677

* webroot -> data when using logo as thumbnail

* Do not list websocket/access token create/delete as integration APIs

* Commit updated API documentation

* Bundle updated admin

* Remove pointing to the 0.0.6 admin branch

* Linter cleanup

* Linter cleanup

* Add donations and follow links to show up under social handles

* Prettified Code!

* More linter cleanup

* Update admin bundle

* Remove use of platforms.js and return icons with social handles. Closes #732

* Update admin bundle

* Support custom config path for use in migration

* Remove unused platform-logos.gif

* Reduce log level of message

* Remove unused logo files in static dir

* Handle dev vs. release build info

* Restore logo.png for initial thumbnail

* Cleanup some files from the build process that are not needed

* Fix incorrect build-time injection var

* Fix missing file getting copied to the build

* Remove console directory message.

* Update admin bundle

* Fix comment

* Report storage setup error

* add some value set error checking

* Use validated dynamic ffmpeg path for animated gif preview

* Make chat message links be white so they don't hide in the bg. Closes #599

* Restore conditional that was accidentally removed

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: nebunez <uoj2y7wak869@opayq.net>
Co-authored-by: gabek <gabek@users.noreply.github.com>
2021-02-18 23:05:52 -08:00

450 lines
12 KiB
Go

package data
import (
"errors"
"sort"
"strings"
"time"
"github.com/owncast/owncast/config"
"github.com/owncast/owncast/models"
log "github.com/sirupsen/logrus"
)
const extraContentKey = "extra_page_content"
const streamTitleKey = "stream_title"
const streamKeyKey = "stream_key"
const logoPathKey = "logo_path"
const serverSummaryKey = "server_summary"
const serverNameKey = "server_name"
const serverURLKey = "server_url"
const httpPortNumberKey = "http_port_number"
const rtmpPortNumberKey = "rtmp_port_number"
const serverMetadataTagsKey = "server_metadata_tags"
const directoryEnabledKey = "directory_enabled"
const directoryRegistrationKeyKey = "directory_registration_key"
const socialHandlesKey = "social_handles"
const peakViewersSessionKey = "peak_viewers_session"
const peakViewersOverallKey = "peak_viewers_overall"
const lastDisconnectTimeKey = "last_disconnect_time"
const ffmpegPathKey = "ffmpeg_path"
const nsfwKey = "nsfw"
const s3StorageEnabledKey = "s3_storage_enabled"
const s3StorageConfigKey = "s3_storage_config"
const videoLatencyLevel = "video_latency_level"
const videoStreamOutputVariantsKey = "video_stream_output_variants"
// GetExtraPageBodyContent will return the user-supplied body content.
func GetExtraPageBodyContent() string {
content, err := _datastore.GetString(extraContentKey)
if err != nil {
log.Errorln(extraContentKey, err)
return config.GetDefaults().PageBodyContent
}
return content
}
// SetExtraPageBodyContent will set the user-supplied body content.
func SetExtraPageBodyContent(content string) error {
return _datastore.SetString(extraContentKey, content)
}
// GetStreamTitle will return the name of the current stream.
func GetStreamTitle() string {
title, err := _datastore.GetString(streamTitleKey)
if err != nil {
return ""
}
return title
}
// SetStreamTitle will set the name of the current stream.
func SetStreamTitle(title string) error {
return _datastore.SetString(streamTitleKey, title)
}
// GetStreamKey will return the inbound streaming password.
func GetStreamKey() string {
key, err := _datastore.GetString(streamKeyKey)
if err != nil {
log.Errorln(streamKeyKey, err)
return ""
}
return key
}
// SetStreamKey will set the inbound streaming password.
func SetStreamKey(key string) error {
return _datastore.SetString(streamKeyKey, key)
}
// GetLogoPath will return the path for the logo, relative to webroot.
func GetLogoPath() string {
logo, err := _datastore.GetString(logoPathKey)
if err != nil {
log.Errorln(logoPathKey, err)
return config.GetDefaults().Logo
}
if logo == "" {
return config.GetDefaults().Logo
}
return logo
}
// SetLogoPath will set the path for the logo, relative to webroot.
func SetLogoPath(logo string) error {
return _datastore.SetString(logoPathKey, logo)
}
// GetServerSummary will return the server summary text.
func GetServerSummary() string {
summary, err := _datastore.GetString(serverSummaryKey)
if err != nil {
log.Errorln(serverSummaryKey, err)
return ""
}
return summary
}
// SetServerSummary will set the server summary text.
func SetServerSummary(summary string) error {
return _datastore.SetString(serverSummaryKey, summary)
}
// GetServerName will return the server name text.
func GetServerName() string {
name, err := _datastore.GetString(serverNameKey)
if err != nil {
log.Errorln(serverNameKey, err)
return ""
}
return name
}
// SetServerName will set the server name text.
func SetServerName(name string) error {
return _datastore.SetString(serverNameKey, name)
}
// GetServerURL will return the server URL.
func GetServerURL() string {
url, err := _datastore.GetString(serverURLKey)
if err != nil {
return ""
}
return url
}
// SetServerURL will set the server URL.
func SetServerURL(url string) error {
return _datastore.SetString(serverURLKey, url)
}
// GetHTTPPortNumber will return the server HTTP port.
func GetHTTPPortNumber() int {
port, err := _datastore.GetNumber(httpPortNumberKey)
if err != nil {
log.Errorln(httpPortNumberKey, err)
return config.GetDefaults().WebServerPort
}
if port == 0 {
return config.GetDefaults().WebServerPort
}
return int(port)
}
// SetHTTPPortNumber will set the server HTTP port.
func SetHTTPPortNumber(port float64) error {
return _datastore.SetNumber(httpPortNumberKey, port)
}
// GetRTMPPortNumber will return the server RTMP port.
func GetRTMPPortNumber() int {
port, err := _datastore.GetNumber(rtmpPortNumberKey)
if err != nil {
log.Errorln(rtmpPortNumberKey, err)
return config.GetDefaults().RTMPServerPort
}
if port == 0 {
return config.GetDefaults().RTMPServerPort
}
return int(port)
}
// SetRTMPPortNumber will set the server RTMP port.
func SetRTMPPortNumber(port float64) error {
return _datastore.SetNumber(rtmpPortNumberKey, port)
}
// GetServerMetadataTags will return the metadata tags.
func GetServerMetadataTags() []string {
tagsString, err := _datastore.GetString(serverMetadataTagsKey)
if err != nil {
log.Errorln(serverMetadataTagsKey, err)
return []string{}
}
return strings.Split(tagsString, ",")
}
// SetServerMetadataTags will return the metadata tags.
func SetServerMetadataTags(tags []string) error {
tagString := strings.Join(tags, ",")
return _datastore.SetString(serverMetadataTagsKey, tagString)
}
// GetDirectoryEnabled will return if this server should register to YP.
func GetDirectoryEnabled() bool {
enabled, err := _datastore.GetBool(directoryEnabledKey)
if err != nil {
return config.GetDefaults().YPEnabled
}
return enabled
}
// SetDirectoryEnabled will set if this server should register to YP.
func SetDirectoryEnabled(enabled bool) error {
return _datastore.SetBool(directoryEnabledKey, enabled)
}
// SetDirectoryRegistrationKey will set the YP protocol registration key.
func SetDirectoryRegistrationKey(key string) error {
return _datastore.SetString(directoryRegistrationKeyKey, key)
}
// GetDirectoryRegistrationKey will return the YP protocol registration key.
func GetDirectoryRegistrationKey() string {
key, _ := _datastore.GetString(directoryRegistrationKeyKey)
return key
}
// GetSocialHandles will return the external social links.
func GetSocialHandles() []models.SocialHandle {
var socialHandles []models.SocialHandle
configEntry, err := _datastore.Get(socialHandlesKey)
if err != nil {
log.Errorln(socialHandlesKey, err)
return socialHandles
}
if err := configEntry.getObject(&socialHandles); err != nil {
log.Errorln(err)
return socialHandles
}
return socialHandles
}
// SetSocialHandles will set the external social links.
func SetSocialHandles(socialHandles []models.SocialHandle) error {
var configEntry = ConfigEntry{Key: socialHandlesKey, Value: socialHandles}
return _datastore.Save(configEntry)
}
// GetPeakSessionViewerCount will return the max number of viewers for this stream.
func GetPeakSessionViewerCount() int {
count, err := _datastore.GetNumber(peakViewersSessionKey)
if err != nil {
return 0
}
return int(count)
}
// SetPeakSessionViewerCount will set the max number of viewers for this stream.
func SetPeakSessionViewerCount(count int) error {
return _datastore.SetNumber(peakViewersSessionKey, float64(count))
}
// GetPeakOverallViewerCount will return the overall max number of viewers.
func GetPeakOverallViewerCount() int {
count, err := _datastore.GetNumber(peakViewersOverallKey)
if err != nil {
return 0
}
return int(count)
}
// SetPeakOverallViewerCount will set the overall max number of viewers.
func SetPeakOverallViewerCount(count int) error {
return _datastore.SetNumber(peakViewersOverallKey, float64(count))
}
// GetLastDisconnectTime will return the time the last stream ended.
func GetLastDisconnectTime() (time.Time, error) {
var disconnectTime time.Time
configEntry, err := _datastore.Get(lastDisconnectTimeKey)
if err != nil {
return disconnectTime, err
}
if err := configEntry.getObject(disconnectTime); err != nil {
return disconnectTime, err
}
return disconnectTime, nil
}
// SetLastDisconnectTime will set the time the last stream ended.
func SetLastDisconnectTime(disconnectTime time.Time) error {
var configEntry = ConfigEntry{Key: lastDisconnectTimeKey, Value: disconnectTime}
return _datastore.Save(configEntry)
}
// SetNSFW will set if this stream has NSFW content.
func SetNSFW(isNSFW bool) error {
return _datastore.SetBool(nsfwKey, isNSFW)
}
// GetNSFW will return if this stream has NSFW content.
func GetNSFW() bool {
nsfw, err := _datastore.GetBool(nsfwKey)
if err != nil {
return false
}
return nsfw
}
// SetFfmpegPath will set the custom ffmpeg path.
func SetFfmpegPath(path string) error {
return _datastore.SetString(ffmpegPathKey, path)
}
// GetFfMpegPath will return the ffmpeg path.
func GetFfMpegPath() string {
path, err := _datastore.GetString(ffmpegPathKey)
if err != nil {
return ""
}
return path
}
// GetS3Config will return the external storage configuration.
func GetS3Config() models.S3 {
configEntry, err := _datastore.Get(s3StorageConfigKey)
if err != nil {
return models.S3{Enabled: false}
}
var s3Config models.S3
if err := configEntry.getObject(&s3Config); err != nil {
return models.S3{Enabled: false}
}
return s3Config
}
// SetS3Config will set the external storage configuration.
func SetS3Config(config models.S3) error {
var configEntry = ConfigEntry{Key: s3StorageConfigKey, Value: config}
return _datastore.Save(configEntry)
}
// GetS3StorageEnabled will return if external storage is enabled.
func GetS3StorageEnabled() bool {
enabled, err := _datastore.GetBool(s3StorageEnabledKey)
if err != nil {
log.Errorln(err)
return false
}
return enabled
}
// SetS3StorageEnabled will enable or disable external storage.
func SetS3StorageEnabled(enabled bool) error {
return _datastore.SetBool(s3StorageEnabledKey, enabled)
}
// GetStreamLatencyLevel will return the stream latency level.
func GetStreamLatencyLevel() models.LatencyLevel {
level, err := _datastore.GetNumber(videoLatencyLevel)
if err != nil || level == 0 {
level = 4
}
return models.GetLatencyLevel(int(level))
}
// SetStreamLatencyLevel will set the stream latency level.
func SetStreamLatencyLevel(level float64) error {
return _datastore.SetNumber(videoLatencyLevel, level)
}
// GetStreamOutputVariants will return all of the stream output variants.
func GetStreamOutputVariants() []models.StreamOutputVariant {
configEntry, err := _datastore.Get(videoStreamOutputVariantsKey)
if err != nil {
return config.GetDefaults().StreamVariants
}
var streamOutputVariants []models.StreamOutputVariant
if err := configEntry.getObject(&streamOutputVariants); err != nil {
return config.GetDefaults().StreamVariants
}
if len(streamOutputVariants) == 0 {
return config.GetDefaults().StreamVariants
}
return streamOutputVariants
}
// SetStreamOutputVariants will set the stream output variants.
func SetStreamOutputVariants(variants []models.StreamOutputVariant) error {
var configEntry = ConfigEntry{Key: videoStreamOutputVariantsKey, Value: variants}
return _datastore.Save(configEntry)
}
// VerifySettings will perform a sanity check for specific settings values.
func VerifySettings() error {
if GetStreamKey() == "" {
return errors.New("no stream key set. Please set one in your config file")
}
return nil
}
// FindHighestVideoQualityIndex will return the highest quality from a slice of variants.
func FindHighestVideoQualityIndex(qualities []models.StreamOutputVariant) int {
type IndexedQuality struct {
index int
quality models.StreamOutputVariant
}
if len(qualities) < 2 {
return 0
}
indexedQualities := make([]IndexedQuality, 0)
for index, quality := range qualities {
indexedQuality := IndexedQuality{index, quality}
indexedQualities = append(indexedQualities, indexedQuality)
}
sort.Slice(indexedQualities, func(a, b int) bool {
if indexedQualities[a].quality.IsVideoPassthrough && !indexedQualities[b].quality.IsVideoPassthrough {
return true
}
if !indexedQualities[a].quality.IsVideoPassthrough && indexedQualities[b].quality.IsVideoPassthrough {
return false
}
return indexedQualities[a].quality.VideoBitrate > indexedQualities[b].quality.VideoBitrate
})
return indexedQualities[0].index
}