Commit graph

24 commits

Author SHA1 Message Date
Gabe Kangas
e81d41d092 Explicitly add unsafe-eval only when running automated browser tests 2021-09-18 10:06:47 -07:00
Gabe Kangas
5f3cecced5 Expand CSP 2021-09-17 02:22:43 -07:00
Christian
1c875243f3
Add hashes for standalone video/chat (#1413) 2021-09-17 02:08:19 -07:00
Gabe Kangas
c6c6f0233d
Expanded linting + fix warnings (#1396)
* Expand the linters and types of warnings to improve consistency and safety

* Fail lint workflow if there are errors

* golint has been replaced by revive

* Hand-pick some of the default exclude list

* Ignore error when trying to delete preview gif

* Ignore linter warning opening playlist path

* Rename user field Id -> ID

* A bunch of renames to address linter warnings

* Rename ChatClient -> Client per linter suggestion best practice

* Rename ChatServer -> Server per linter suggestion best practice

* More linter warning fixes

* Add missing comments to all exported functions and properties
2021-09-12 00:18:15 -07:00
Gabe Kangas
7278ce8f26
Merge pull request from GHSA-2hfj-cxw7-g45p 2021-08-30 19:43:28 -07:00
Yarmo Mackenbach
cab963f21c
Fix status code for options requests (#1290) 2021-08-03 12:23:20 -07:00
Gabe Kangas
810b0cd5da Explicitly add CORS wildcard on all OPTIONS requests 2021-07-28 12:47:15 -07:00
Gabe Kangas
509c658080 Add OPTIONS preflight support for 3rd party auth.
- Explicitly add wildcard CORS header within the middleware.
- Accept all OPTIONS preflight requests within the middlware.
- Add success tests for the OPTIONS request.
- Add failure tests for GET requests.
2021-07-28 12:40:55 -07:00
Gabe Kangas
b6f68628c0
Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
2021-07-19 19:22:29 -07:00
Gabe Kangas
8287484cc6 Add Google FLoC opt-out header on web page requests. Closes #939 2021-04-15 22:25:07 -07:00
Gabe Kangas
bc2caadb74
0.0.6 -> Master (#731)
* Implement webhook events for external integrations (#574)

* Implement webhook events for external integrations

Reference #556

* move message type to models and remove duplicate

* add json header so content type can be determined

* Pass at migrating webhooks to datastore + management apis (#589)

* Pass at migrating webhooks to datastore + management apis

* Support nil lastUsed timestamps and return back the new webhook on create

* Cleanup from review feedback

* Simplify a bit

Co-authored-by: Aaron Ogle <aaron@geekgonecrazy.com>

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Webhook query cleanup

* Access tokens + Send system message external API (#585)

* New add, get and delete access token APIs

* Create auth token middleware

* Update last_used timestamp when using an access token

* Add auth'ed endpoint for sending system messages

* Cleanup

* Update api spec for new apis

* Commit updated API documentation

* Add auth'ed endpoint for sending user chat messages

* Return access token string

* Commit updated API documentation

* Fix route

* Support nil lastUsed time

* Commit updated Javascript packages

* Remove duplicate function post rebase

* Fix msg id generation

* Update controllers/admin/chat.go

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Webhook query cleanup

* Add SystemMessageSent to EventType

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Set webhook as used on completion. Closes #610

* Display webhook errors as errors

* Commit updated API documentation

* Add user joined chat event

* Change integration API paths. Update API spec

* Update development version of admin that supports integration apis

* Commit updated API documentation

* Add automated tests for external integration APIs

* check error

* quiet this test for now

* Route up some additional 3rd party apis. #638

* Commit updated API documentation

* Save username on user joined event

* Add missing scope to valid scopes list

* Add generic chat action event API for 3rd parties. Closes #666

* Commit updated API documentation

* First pass at moving WIP config framework into project for #234

* Only support exported fields in custom types

* Using YP get/set key as a first pass at using the data layer. Fixes + integration.

* Ignore test db

* Start adding getters and setters for config values

* More get/set config work. Starting to populate api with data

* Wire up some config edit endpoints

* More endpoints

* Disable cors middleware

* Add more endpoints and add test to test them

* Remove the in-memory change APIs

* Add endpoint for changing tags

* Add more config endpoints

* Starting to point more things away from config file and to the datastore

* Populate YP with db data

* Create new util method for parsing page body markdown and return it in api

* Verify proposed path to ffmpeg

* For development purposes show the config key in logs

* Move stats values to datastore

* Moving over more values to the datastore

* Move S3 config to datastore

* First pass the config -> db migrator

* Add the start of the video config apis

* It builds pointing everything away from the config

* Tweak ffmpeg path error message

* Backup database every hour. Closes #549

* Config + defaults + migration work for db

* Cleanup logging

* Remove all the old config structs

* Add descriptive info about migration

* Tweak ffmpeg validation logic

* Fix db backup path. backup on db version migration

* Set video and s3 configurations

* Update api spec with new config endpoints

* Add migrator for stats file

* Commit updated API documentation

* Use a dynamic system port for internal HLS writes. Closes #577 (#626)

* Use a dynamic system port for internal HLS writes. Closes #577

* Cleanup

* YP key migration to datastore

* Create a backup directory if needed before migrations

* Remove config test that no longer makes sense. Cleanup.

* Change number types from float32 to float64

* Update automated test suite

* Allow restoring a database backup via command line flags. Closes #549

* Add new hls segment config api

* Commit updated API documentation

* Update apis to require a value container property

* add socialHandles api

* Commit updated API documentation

* Add new latancy level setting to replace segment settings

* Commit updated API documentation

* Fix spelling

* Commit updated API documentation

* hardcode a json api of available social platforms

* Add additional icons

* Return social handles in server config api

* Add socialhandles validation to test

* Move list of hard coded social platforms to an api

* Remove audio only code from transcoder since we do not use it

* Add latency levels api + snapshot of video settings as current broadcast

* Add config/serverurl endpoint

* Return 404 on YP api if disabled

* Surface stream title in YP response

* Add stream title to web ui

* Cleanup log message. Closes #520

* Rename ffmpeg package to transcoder

* Add ws package for testing

* Reduce chat backlog to past 5hrs, max 50. Closes #548

* Fix error formatting

* Add endpoint for resetting yp registration

* Add yp/reset to api spec. return status in response

* Return zero viewer count if stream is offline. Closes #422

* Post-rebase fixes

* Fix merge conflict in openapi file

* Commit updated API documentation

* Standardize controller names

* Support setting the stream key via the command line. Closes #665

* Return social handles with YP data. First half of https://github.com/owncast/owncast-yp/issues/28

* Give the YP package access to server status regardless if enabled or not

* Change delay in automated tests

* Add stream title integration API. For #638

* Commit updated API documentation

* Add storage to the migrator

* Missing returning NSFW value in server config

* Add flag to ignore websocket client. Closes #537

* Add error for parsing broadcaster metadata

* Add support for a cli specified http server port. Closes #674

* Add cpu usage levels and a temporary mapping between it and libx264 presets

* Test for valid url endpoint when saving s3 config

* Re-configure storage on every stream to allow changing storage providers

* After 5 minutes of a stream being stopped clear the stream title

* Hide viewer count once stream goes offline instead of when player stops

* Pull steamTitle from the status that gets updated instead of the config

* Commit updated API documentation

* Optionally show stream title in the header

* Reset stream title when server starts

* Show chat action when stream title is updated

* Allow system messages to come back in persistence

* Split out getting chat history for moderation + fix tests

* Remove server title and standardize on name only

* Commit updated API documentation

* Bump github.com/aws/aws-sdk-go from 1.37.1 to 1.37.2 (#680)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.1 to 1.37.2.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.1...v1.37.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add video variant and stream latency config file migrator

* Remove mostly unused disable upgrade check bool

* Commit updated API documentation

* Allow bundling the admin from the 0.0.6 branch

* Fix saving port numbers

* Use name instead of old title on window focus

* Work on latency levels. Fix test to use levels. Clean up transcoder to only reference levels

* Another place where title -> name

* Fix test

* Bump github.com/aws/aws-sdk-go from 1.37.2 to 1.37.3 (#690)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.2 to 1.37.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.2...v1.37.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update dependabot config

* Bump github.com/aws/aws-sdk-go from 1.37.3 to 1.37.5 (#693)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.3 to 1.37.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.3...v1.37.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript (#694)

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript

Bumps [video.js](https://github.com/videojs/video.js) from 7.10.2 to 7.11.4.
- [Release notes](https://github.com/videojs/video.js/releases)
- [Changelog](https://github.com/videojs/video.js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/videojs/video.js/compare/v7.10.2...v7.11.4)

Signed-off-by: dependabot[bot] <support@github.com>

* Commit updated Javascript packages

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>

* Make the latency migrator dynamic so I can tweak values easier

* Split out fetching ffmpeg path from validating the path so it can be changed in the admin

* Some commenting and linter cleanup

* Validate the path for a logo change and throw an error if it does not exist

* Logo change requests have to be a real file now

* Cleanup, making linter happy

* Format javascript on push

* Only format js in master

* Tweak latency level values

* Remove unused config file examples

* Fix thumbnail generation after messing with the ffmpeg path getter

* Reduce how often we report high hardware utilization warnings

* Bundle the 0.0.6 branch version of the admin

* Return validated ffmpeg path in admin server config

* Change the logo to be stored in the data directory instead of webroot

* Bump postcss from 8.2.4 to 8.2.5 in /build/javascript (#702)

Bumps [postcss](https://github.com/postcss/postcss) from 8.2.4 to 8.2.5.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.2.4...8.2.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Default config file no longer used

* don't show stream title when offline

addresses https://github.com/owncast/owncast/issues/677

* Remove auto-clearing stream title. #677

* webroot -> data when using logo as thumbnail

* Do not list websocket/access token create/delete as integration APIs

* Commit updated API documentation

* Bundle updated admin

* Remove pointing to the 0.0.6 admin branch

* Linter cleanup

* Linter cleanup

* Add donations and follow links to show up under social handles

* Prettified Code!

* More linter cleanup

* Update admin bundle

* Remove use of platforms.js and return icons with social handles. Closes #732

* Update admin bundle

* Support custom config path for use in migration

* Remove unused platform-logos.gif

* Reduce log level of message

* Remove unused logo files in static dir

* Handle dev vs. release build info

* Restore logo.png for initial thumbnail

* Cleanup some files from the build process that are not needed

* Fix incorrect build-time injection var

* Fix missing file getting copied to the build

* Remove console directory message.

* Update admin bundle

* Fix comment

* Report storage setup error

* add some value set error checking

* Use validated dynamic ffmpeg path for animated gif preview

* Make chat message links be white so they don't hide in the bg. Closes #599

* Restore conditional that was accidentally removed

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: nebunez <uoj2y7wak869@opayq.net>
Co-authored-by: gabek <gabek@users.noreply.github.com>
2021-02-18 23:05:52 -08:00
Gabe Kangas
6df6b97815 Log auth middleware failures as Debug. Closes #342 2020-11-12 15:37:20 -08:00
Christian Muehlhaeuser
2e1f8d29b5
Fix godoc style comments (#356) 2020-11-12 15:14:59 -08:00
Gabe Kangas
5c6f5fc697 Add logging admin APIs for dashboard #114 2020-10-29 18:17:04 -07:00
Gabe Kangas
6ea9affce0
HLS video handling/storage/state refactor (#151)
* WIP with new transcoder progress monitor

* A whole different WIP in progress monitoring via local PUTs

* Use an actual hls playlist parser to rewrite master playlist

* Cleanup

* Private vs public path for thumbnail generation

* Allow each storage provider to make decisions of how to store different types of files

* Simplify inbound file writes

* Revert

* Split out set stream as connected/disconnected state methods

* Update videojs

* Add comment about the hls handler

* Rework of the offline stream state.  For #85

* Delete old unreferenced video segment files from disk

* Cleanup all segments and revert to a completely offline state after 5min

* Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup.

* Update transcoder test

* Add comment

* Return http 200 on success to transcoder. Tweak how files are written to disk

* Force pixel color format in transcoder

* Add debugging info for S3 transfers. Add default ACL.

* Fix cleanup timer

* Reset session stats when we cleanup the session.

* Put log file back

* Update test

* File should not be a part of this commit

* Add centralized shared performance timer for use anywhere

* Post-rebase cleanup

* Support returning nil from storage provider save

* Updates to reflect package changes + other updates in master

* Fix storage providers being overwritten

* Do not return pointer in save. Support cache headers with S3 providers

* Split out videojs + vhs and point to specific working versions of them

* Bump vjs and vhs versions

* Fix test

* Remove unused

* Update upload warning message

* No longer valid comment

* Pin videojs and vhs versions
2020-10-14 14:07:38 -07:00
Gabe Kangas
268085e41f I thought I already fixed this. Guess it got lost in a merge. 2020-10-07 20:53:01 -07:00
Gabe Kangas
0445274fbd Do not cache hls playlist files at all. Closes #229 2020-10-06 18:26:11 -07:00
Gabe Kangas
06e5fe09f4 Fix leftover value when we would deal with cache duration in minutes instead of seconds 2020-10-06 09:17:41 -07:00
leowmjw
769fc3a561
Fixes #211: Update mod paths (#221)
Co-authored-by: Michael Leow <mleow@moneylion.com>
2020-10-05 10:07:09 -07:00
Gabe Kangas
bb9c788306 Support CORS+Basic auth together 2020-10-03 23:06:48 -07:00
Gabe Kangas
7b64fc7c30
Disconnect stream Admin API + HTTP Basic Auth (#204)
* Create http auth middleware

* Add support for ending the inbound stream. Closes #191

* Add a simple success response to API requests
2020-10-01 18:16:58 -07:00
Gabe Kangas
8aa5c33999
Update caching logic/headers (#184)
* Improve caching by adding etags and reducing the max-age

* Move caching into middleware. Set cache headers on segments
2020-09-30 14:14:39 -07:00
Gabe Kangas
351fbe8834 Set a short 1 day cache duration until we do something more complex. Closes #131 2020-08-30 16:07:20 -07:00
Bradley Hilton
487bd12444
Project restructure (#18)
* First pass at restructuring the project; untested but it does compile

* Restructure builds and runs 🎉

* Add the dist folder to the gitignore

* Update core/playlist/monitor.go

* golint and reorganize the monitor.go file

Co-authored-by: Gabe Kangas <gabek@real-ity.com>
2020-06-22 18:11:56 -07:00