Commit graph

369 commits

Author SHA1 Message Date
Gabe Kangas
3585340f1b Update test 2021-11-23 22:02:39 -08:00
Eng Zer Jun
7996df85e1
refactor: move from io/ioutil to io and os packages (#1546)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-19 22:42:50 -08:00
Jannik
da199e5775
use worker pool to limit webhooks to 10 concurrent http executions (#1510) (#1525)
* refactor: use worker pool to limit webhooks to 10 concurrent http executions (#1510)

* chore: try to please go linter
2021-11-14 10:02:52 -08:00
Gabe Kangas
c9f8133507 Fix incorrect webhook payload being sent in the new SetMessageVisibilityEvent 2021-11-12 16:28:53 -08:00
Gabe Kangas
6ffa020c5f Updated linter threw new warnings. Fixed. 2021-11-03 09:43:47 -07:00
gingervitis
9a91324456
Inline chat moderation UI (#1331)
* - mock detect when user turns into moderator
- add moderator indicator to display on messages and username changer

* also mock moderator flag in message payload about user to display indicator

* add some menu looking icons and a menu of actions

* WIP chat moderators

* Add support for admin promoting a user to moderator

* WIP-
open a more info panel of user+message info; add some a11y to buttons

* style the details panel

* adjust positioning of menus

* Merge fixes. ChatClient->Client ChatServer->Server

* Remove moderator bool placeholders to use real state

* Support inline hiding of messages by moderators

* Support inline banning of chat users

* Cleanup linter warnings

* Puppeteer tests fail after typing take place

* Manually resolve conflicts in chat between moderator feature and develop

Co-authored-by: Gabe Kangas <gabek@real-ity.com>
2021-11-02 19:27:41 -07:00
Gabe Kangas
b43c5e674e
Rework how hiding messages works. (#1509)
* Rework how hiding messages work. Fixes #1350

* Remove unused function

* Revert to old event name to support previously saved webhooks
2021-11-02 18:00:15 -07:00
Jannik
2278fec70a
execute webhooks in parallel (and 🐛 update webhooks.last_used correctly) (#1505)
*  execute webhooks in parallel

* 🐛 update the webhooks.last_used column correctly
2021-11-01 19:13:20 -07:00
Gabe Kangas
4c37308e71 Update the HLS flags to adhere to more Apple recommendations 2021-11-01 00:20:08 -07:00
Gabe Kangas
5b38426e31 Tweak video bandwidth values to be < 10% over max 2021-10-31 23:00:11 -07:00
Jack
651caeba91
Fix build for Windows (#1377) (#1506)
* Fix build for Windows (#1377)

* Add tests for windows
2021-10-31 14:01:22 -07:00
Jack
f0a11cac5d
Optimise regexp usage (#1502)
* Optimise regexp usage

* Revert regexp removal

* Atleast use the semioptimised version
2021-10-29 12:29:01 -07:00
Artur Angiel
2110dfd30c
Support path-based S3 storage. Closes #1495 (#1496)
* Support path-based S3 storage. Closes #1495

Revert "Remove forcing old path-style URLs with s3. Closes #497"

This reverts commit b2953028cf.

* #1495 Path-style S3 compatibile URLs implemented

https://github.com/owncast/owncast/issues/1495

It gives ability to use S3 compatibile providers that doesn't
support virtual-host-style URLs (i.e. Oracle Cloud Object Storage)

Co-authored-by: Artur Angiel <artur@angiel.ovh>
2021-10-28 17:27:44 -07:00
Gabe Kangas
bdce2e13bf Some linter cleanup 2021-10-25 00:31:45 -07:00
Gabe Kangas
3f4176d8d6 Replace panics with our logging framework 2021-10-25 00:15:58 -07:00
Gabe Kangas
c4d2ffd834
Fix crash when an invalid inbound stream is disconnected. Closes #1439 (#1483) 2021-10-21 17:21:17 -07:00
Tim Cooper
12eb59f611
refactor geoip (#1442)
- Introduce a new Client type to remove the global variables from the file
- Use the sync package to prevent race conditions with the cache and
  enabled flag
- Cache results for IPs, even if the result is nil

There are still data races around the client.Geo variable, but that can be
resolved in a future commit.
2021-10-12 13:21:37 -07:00
Tim Cooper
9a912d5385
remove global variables from core/storageproviders (#1444) 2021-10-11 16:29:36 -07:00
Gabe Kangas
ca9d5de192
Embed static resources (#1466)
* Replace pkger with go:embed for bundling the admin. Closes #844

* Remove references to pkged.go

* Point tests to use an updated version of Go

* Add comment to new exported function

* Cleanup

* Add a dummy pkged.go to alert people to stop using it.

* Add simple browser test to make sure the admin is available and renders

* Don't panic

* Embed bot/scraper metadata template.

Add browser test to validate the rendering of this template.

* Use embedded offline.ts segment

* Remove placeholder thumbnail as its unnecessary

* Remove copying the static directory into the release

* Cleanup
2021-10-11 15:04:16 -07:00
Gabe Kangas
fb1c919e59
Fix the destination path. Closes #1455 (#1456) 2021-10-05 16:45:39 -07:00
Gabe Kangas
db22931fb5
Do not send user joined messages for already active users (#1416)
* Do not send user joined messages for active users

* Reduce from active within 10min to 5
2021-09-21 14:06:23 -07:00
Ruffy
9c484efd36
Feature/oc 1316 add support for system message to single user (#1351)
* add public func to lookup a ChatClient by its clientId

* add facility to send a system message directly to a user

* add clientId field to UserEvent

* implement simple http endpoint to send a message to a user

* let mux handle new directSystemMessageToUser endpoint

* add ClientId to UserEvents across the codebase

* render body of system-message to client

* add clientId to Chat-Message

* add tests showing how url-parsing should work

* add simple rest endpoint helpers for parameter-parsing and easy routing

* use newly added rest-endpoint helper to rout to Client-Messaging controller

* use safe "ReadRestUrlParameter" to parse ClientId

* remove empty HandleFunc in router

* set Header directly to prevent built-in (platform-dependent) canonicalization to kick in

* fix typo in "Parameter" message

* remove debug-logging of HTTP headers in REST-helpers

* convert to uint32 to prevent overruns when converting to wraptype uint later on

* resolve linter-ouchies

* resolve linter potential nil-deref warning

* document the SendSystemMessageToClient endpoint in swaggerdoc

* remove clientId assignment causing potential nil dereference in userDisabledEvent-case

as the clientId isn't relevant here anyway

* make findClientById private, so its not accessible outside of core/chat

* remove redundant string type hint

* Update PR based on linter requirements

Co-authored-by: Raffael Rehberger <raffael@rtrace.io>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>
2021-09-13 01:26:28 -07:00
Gabe Kangas
1b053ffd1b
Simplify HLS storage paths (#1393)
* Remove private vs public HLS paths and add a HLS controller. Closes #875

* Use http.ServeFile instead
2021-09-12 11:32:42 -07:00
Gabe Kangas
c6c6f0233d
Expanded linting + fix warnings (#1396)
* Expand the linters and types of warnings to improve consistency and safety

* Fail lint workflow if there are errors

* golint has been replaced by revive

* Hand-pick some of the default exclude list

* Ignore error when trying to delete preview gif

* Ignore linter warning opening playlist path

* Rename user field Id -> ID

* A bunch of renames to address linter warnings

* Rename ChatClient -> Client per linter suggestion best practice

* Rename ChatServer -> Server per linter suggestion best practice

* More linter warning fixes

* Add missing comments to all exported functions and properties
2021-09-12 00:18:15 -07:00
Halil İbrahim Kocaöz
d90b02e50b
fix moved transcoding error links (#1381) 2021-09-05 18:07:02 -07:00
ooa113y
efb80243d5
Fix Windows tests (#1376)
* Fix Windows tests

* Use filepath.Join
2021-09-04 21:17:15 -07:00
Gabe Kangas
e3b0af1b67 Expand upon flood detection and chat rate limiting. Closes #1349 2021-08-27 14:43:09 -07:00
Gabe Kangas
1a1e2a3738 Revert "Update test to reflect the newline -> br tag change"
This reverts commit 3d09698355.
2021-08-18 18:08:17 -07:00
Gabe Kangas
e5735b8d6d Remove adding <br> tags to newlines 2021-08-18 18:08:07 -07:00
Gabe Kangas
3d09698355 Update test to reflect the newline -> br tag change 2021-08-18 17:51:33 -07:00
Gabe Kangas
3618a28240 Re-enable unsafe tags as they are needed for emoji 2021-08-18 17:00:52 -07:00
Gabe Kangas
8907566ed1 Fixing chat message formatting in #1315 2021-08-18 16:11:00 -07:00
Gabe Kangas
04bb97bffc Fix build error on FreeBSD. Closes #1243 2021-08-13 15:34:00 -07:00
Gabe Kangas
4ea416fdbe Update the user object name change time when changing name. Closes #1277 2021-08-13 00:05:22 -07:00
Gabe Kangas
6f57f570d2 Restore GeoIP support to now support chat users. Closes #1304 2021-08-13 08:52:36 +02:00
Gabe Kangas
6fb383d04b Only ignore messages 5min after disconnect. Part of fixing #1268 2021-08-12 22:40:10 -07:00
Gabe Kangas
f87d4f960a Add formatting 2021-08-12 22:39:12 -07:00
Gabe Kangas
cf87f53da3 Do not send empty type when its not apropriate 2021-08-12 21:20:06 -07:00
Gabe Kangas
9d0ae44c3c Give stream status webhooks an id and timestamp 2021-08-12 20:02:17 -07:00
Gabe Kangas
53d0e8653e Remove old unused function 2021-08-12 17:23:28 -07:00
Gabe Kangas
b966d32d77 Incorrect webhook event type is being sent on user joined. Fixes #1302 2021-08-12 13:19:09 -07:00
Gabe Kangas
dcc7a7c5c3 Support assigning nil as time instead of setting time as invalid. Fix typo that led to incorrect disconnect time. Maybe addresses #1268? 2021-08-03 17:47:16 -07:00
Meisam
41a7e8b896
check the return value of os.Rename (#1288) 2021-07-31 14:59:02 -07:00
Gabe Kangas
12cbe529de Fix incorrect handling of messages array. Closes #1287 2021-07-31 12:48:42 -07:00
Christian
cb7a9b89ea
safely generate the thumbnail.jpg & preview.gif (#1279) 2021-07-28 14:21:02 -07:00
Gabe Kangas
031a848b7a No need to publicly log this 2021-07-28 12:37:26 -07:00
Meisam
109d2669ab
better chat message sanitization (#1266)
* strip <p> in chat sanitization, keep the content

* update sanitization tests

* update tests

* rm <p></p> comparison for empty messages
2021-07-27 15:26:27 -07:00
Meisam
44f406caf0
reject user messages when offline (#1269) 2021-07-27 10:42:05 -07:00
Gabe Kangas
fac06257ad Add in the optimization where multiple websocket events can exist within a single message 2021-07-26 19:23:15 -07:00
Gabe Kangas
8b2747e4d7 Add some sqlite optimizations 2021-07-26 17:35:23 -07:00
Gabe Kangas
f20aa4478d Add a simple caching of chat history as an optimization 2021-07-26 17:35:04 -07:00
Gabe Kangas
f85b54cfeb Pass along just the client id instead a reference to the entire client when unregistering the client 2021-07-26 17:34:50 -07:00
Meisam
f07c9e2e00
consider <p></p> as an empty message (#1264) 2021-07-26 09:21:48 -07:00
Christian Muehlhaeuser
35a0c6fa14
Improve performance of retrieving chat history by avoiding double-sort and outer join (#1261) 2021-07-25 09:54:24 -07:00
Meisam
e1c4b452e6
test emoji title and alt tag sanitization (#1244) 2021-07-23 11:01:30 -07:00
Meisam
a8e93de134
Prevent remote image injection with /img/emoji/ in url (#1245)
* test remote img blocking with /img/emoji/ in url

* fix emoji filter

prevent injection of remote img with /img/emoji in url
2021-07-23 11:00:04 -07:00
Gabe Kangas
ae78283caf Remove extra log from displaying 2021-07-22 23:34:51 -07:00
Gabe Kangas
484098afda Some migration fixes and database optimizations 2021-07-22 23:30:25 -07:00
Meisam
c4c1ecfc7b
Allow alt and title tags in chat emojis (#1241)
* allow alt and title on chat imgs

* enforce non-empty alt&title tags for emojis
2021-07-22 22:22:33 -07:00
Gabe Kangas
3dc3ad75f6 Cleanup errors that are not fatal during filesystem cleanup 2021-07-22 16:50:30 -07:00
Gabe Kangas
c3e8e78dad Centralize chan closure to be done in the client, not the server. Set chan size to max message size. 2021-07-22 15:27:12 -07:00
Gabe Kangas
f782e82909 Fix potential concurrent access condition 2021-07-22 15:27:12 -07:00
Gabe Kangas
44d6a36b77 Cleanup linter warnings 2021-07-19 23:49:16 -07:00
Gabe Kangas
b6f68628c0
Chat refactor + persistent backing chat users (#1163)
* First pass at chat user registration and validation

* Disable chat if the user is disabled/blocked or the server hits max connections

* Handle dropping sockets if chat is disabled

* Fix origin in automated chat test

* Work for updated chat moderation

* Chat message markdown rendering and fix tests

* Put /api/chat behind a chat user access token. Closes #1085

* Reject blocked username changes

* More WIP moderation

* Defer configuring chat until we know if it is enabled. Closes #1135

* chat user blocking. Closes #1096

* Add tests around user access for #1096

* Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092

* Delete old chat messages from db as to not hold on to excessive data. Closes #1152

* Add schema migration for messages. Closes #1155

* Commit updated API documentation

* Add chat load test

* Shared db mutex and db optimizations

* Simplify past display name handling

* Use a new test db for each test run

* Wire up the external messages actions + add tests for them

* Move access tokens to be actual users

* Run message pruning at launch + fix comparison

* Do not return API users in disabled users response

* Fix incorrect highlighting. Closes #1160

* Consolidate user table statements

* Set the max process connection limit to 70% of maximum

* Fix wrong old display name being returned in name change event

* Delete the old chat server files

* Wire back up the webhooks

* Remove unused

* Invalidate user cache on changes

* Do not send rendered body as RawBody

* Some cleanup

* Standardize names for external API users to ExternalAPIUser

* Do not log token

* Checkout branch when building admin for testing

* Bundle in dev admin for testing

* Some cleanup

* Cleanup js logs

* Cleanup and standardize event names

* Clean up some logging

* Update API spec. Closes #1133

* Commit updated API documentation

* Change paths to be better named

* Commit updated API documentation

* Update admin bundle

* Fix duplicate event name

* Rename scope var

* Update admin bundle

* Move connected clients controller into admin package

* Fix collecting usernames for autocomplete purposes

* No longer generate username when it is empty

* Sort clients and users by timestamp

* Move file to admin controller package

* Swap, so the comments stay correct

Co-authored-by: Jannik <jannik@outlook.com>

* Use explicit type alias

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented code.

Co-authored-by: Jannik <jannik@outlook.com>

* Cleanup test

* Remove some extra logging

* Add some clarity

* Update dev instance of admin for testing

* Consolidate lines

Co-authored-by: Jannik <jannik@outlook.com>

* Remove commented unused vars

Co-authored-by: Jannik <jannik@outlook.com>

* Until needed do not return IP address with client list

* Fix typo of wrong var

* Typo led to a bad test. Fix typo and fix test.

* Guard against the socket reconnecting on error if previously set to shutdown

* Do not log access tokens

* Return success message on enable/disable user

* Clean up some inactionable error messages. Sent ban message. Sort banned users.

* fix styling for when chat is completely disabled

* Unused

* guard against nil clients

* Update dev admin bundle

* Do not unhide messages when unblocking user just to be safe. Send removal action from the controller

* Add convinience function for getting active connections for a single user

* Lock db on these mutations

* Cleanup force disconnect using GetClientsForUser and capture client reference explicitly

* No longer re-showing banned user messages for safety. Removing this test.

* Remove no longer needed comment

* Tweaks to forbidden username handling.

- Standardize naming to not use "block" but "forbidden" instead.
- Pass array over the wire instead of string.
- Add API test
- Fix default list incorrectly being appended to custom list.

* Logging cleanup

* Update dev admin bundle

* Add an artificial delay in order to visually see message being hidden when testing

* Remove the user cache as it is a premature optimization

* When connected to chat let the user know their current user details to sync the username in the UI

* On connected send current display name back to client.
- Move name change out of chat component.
- Add additional event type constants.

* Fix broken workflow due to typo

* Troubleshoot workflow

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181)

* Bump htm from 3.0.4 to 3.1.0 in /build/javascript

Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0.
- [Release notes](https://github.com/developit/htm/releases)
- [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0)

---
updated-dependencies:
- dependency-name: htm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* Run npm run build and update libraries

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Commit updated Javascript packages

* Re-send current user info when a rejected name change takes place

* All socket writes should be through the send chan and not directly

* Seed the random generator

* Add keys and indexes to users table

* a util to generate consistent emoji markup

* console clean up

* mod tidy

* Commit updated API documentation

* Handle the max payload size of a socket message.
- Only close socket if x2 greater than the max size.
- Send the user a message if a message is too large.
- Surface the max size in bytes in the config.

* Update admin bundle

* Force all events to be sent in their own socket message and do not concatinate in a single message

* Update chat embed to register for access token

* Use a different access token for embed chat

* Update the chat message bubble background color to be bolder

* add base tag to open links in new window, closes #1220

* Support text input of :emoji: in chat (#1190)

* Initial implementation of emoji injection

* fix bookkeeping with multiple emoji

* make the emoji lookup case-insensitive

* try another solution for Caretposition

* add title to emojis

minor refactoring

* bind moji injection to InputKeyUp

* simplify the code

replace all found emojis

* inject emoji if the modifer is released earlier

* more efficient emoji tag search

* use json emoji.emoji as url

* use createEmojiMarkup()

* move emojify() to chat.js

* emojify on paste

* cleanup emoji titles in paste

* update inputText in InputKeyup

* mark emoji titles with 2*zwnj

this way paste cleanup will not interfere with text which include zwnj

* emoji should not change the inputText

* Do not show join messages when chat is offline. Closes #1224
- Show stream starting/ending messages in chat.
- When stream starts show everyone the welcome message.

* Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222

* use maxSocketPayloadSize to calculate total bytes of message payload (#1221)

* utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting

* add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Jannik <jannik@outlook.com>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
2021-07-19 19:22:29 -07:00
Gabe Kangas
37fedb8ab8 Ignore some additional non-fatal errors 2021-07-18 13:10:08 -07:00
Meisam
7361578412
style fix for error checking (#1170)
+additional linting
2021-07-09 11:16:44 -07:00
Meisam
a13e1e75e2
proper cleanup and permission check for the hls directory (#1167)
* check error on hls cleanup

* rm HLS directories before creating new ones

* don't mask the variable

* mv cleanupDirectory() to utils

* add user-friendly error messages
2021-07-08 12:35:53 -07:00
Jannik
3f9f4a151c
refactor: use io.Pipe and ffmpeg's stdin (#1148)
This removes the usage of `syscall.Mkfifo` which was previously used and
won't work on Windows systems and opens the door for other processes on
the computer to interfere in the rtmp stream (dumping bad content in the
fifo, removing the file, blocking the file in offline status).
Instead, this patch introduces an `io.Pipe` which pipes the RTMP stream
to the ffmpeg command while staying in Owncast.

Further links:
* ffmpeg on using `pipe:0` as an input: https://ffmpeg.org/ffmpeg-protocols.html#pipe
2021-07-03 12:28:25 -07:00
Jannik
edc777ae1b
🐛 use currentBroadcast for playlist overwrites (#1149)
Fixes #1147 since it looks into currentBroadcast and does not use the configuration of the next stream.
2021-07-01 18:27:56 -07:00
Gabe Kangas
0802a20f4f Do not save invalid disconnect time 2021-06-29 11:38:13 -07:00
Gabe Kangas
12104978e8 Some linter cleanup 2021-06-29 10:21:00 -07:00
Gabe Kangas
ab3bbd59bf Cleanup the persisted disconnected time 2021-06-29 10:04:26 -07:00
Gabe Kangas
d28c4b96f3 Fix invalid nil time being returned 2021-06-28 13:59:23 -07:00
Gabe Kangas
1586d80a32 Wire up blocked chat username controller 2021-06-21 20:16:21 -07:00
Gabe Kangas
20c272ff3d Guard against invalid last disconnect time 2021-06-20 20:16:27 -07:00
Gabe Kangas
89dfcfb047 Fix buggy saving of lastDisconnectTime now used for #1111 2021-06-20 11:30:29 -07:00
Meisam
e991199735
updates to backups directory (#1099)
* read BackupDirectory from command line flag

* Change the default backup directory

* mkdir BackupDirectory

* use config for backup file path

* migrateDatabase to the backup directory

* use DoesFileExists

change permission on the directory to 0700

* declare err

* generate backupFile  where needed

* style fix

* more style fixes

* more style fixes
2021-06-13 15:28:59 -07:00
Gabe Kangas
306a0066d1 Fix concurrency crash. Closes #1067 2021-06-07 21:59:43 -07:00
Jannik
fae2c58259
Fix rtmp secret validation to allow / (#1069) (#1070)
* Fix rtmp secret validation to allow `/` (#1069)

* add negative test cases for stuff before /live/

* simplify since Url.Path is already stripping the host

This means that we can simplify the code and make it much clearer.
Removes the tests that checked for the host and stuff between the host and /live/.
2021-06-04 20:09:43 -07:00
Gabe Kangas
94136d4361 Demote log message 2021-05-24 21:43:13 -07:00
leuc
5ab901bb36
Fix #981 Use -webserverip to set http listen address (#1032)
* Fix #981 Use -webserverip to set http listen address

* use 0.0.0.0 as default http listen address

* add Admin REST API for setting http listen address

* full input validation of port and IP
2021-05-24 16:13:49 -07:00
Gabe Kangas
1a0beb1d0e Fix tests 2021-05-23 14:27:35 -07:00
Meisam
8b57ddcfb1
Remove config file migrator (#1045)
* rm migrator consts

* Delete migrator.go

* rm RunMigrations()
2021-05-23 13:36:34 -07:00
Fabian Fischer
d3696cd0aa
fix: avoid duplicate cache-control header (#1048) 2021-05-23 13:35:05 -07:00
Gabe Kangas
61e07bf945 API for setting a list of blocked usernames. For #782 2021-05-22 21:50:34 -07:00
Gabe Kangas
83ad6db394 If tag storage is an empty string return back an empty slice. Fixes #974 2021-05-22 21:32:56 -07:00
Gabe Kangas
381daaec9b Save owncast logs to log file. Closes #908 2021-05-22 19:25:54 -07:00
Gabe Kangas
36a15a97dd Handle the server summary data as auto-linked markdown. Closes #1010 2021-05-22 17:09:57 -07:00
tomleb
1504ea3509
Add -logdir flag (#1039)
This allow configuring the various logs that may be outputted
(transcoder and future logs)
2021-05-22 16:29:49 -07:00
Gabe Kangas
91f6dcd0f6
Active viewer PING controller. Closes #790 (#990) 2021-05-20 20:29:01 -07:00
Gabe Kangas
3810ce4f63 Do not try to copy logo if image does not exist on disk 2021-05-14 15:28:13 -07:00
Gabe Kangas
09da85090e Fix tests 2021-05-05 19:58:02 -07:00
Gabe Kangas
4f4cef97e2 Force an iframe every segment 2021-05-05 18:21:27 -07:00
Gabe Kangas
fc123ecda3 Fix tests 2021-05-05 17:31:19 -07:00
Gabe Kangas
6bcbe60c28 Update latency level values 2021-05-05 17:22:57 -07:00
Gabe Kangas
fa3a959dc5 update tests 2021-04-29 22:13:47 -07:00
Gabe Kangas
5324d235bf Force an i-frame every second for consistent segment lenghts with
minimum latency.
2021-04-29 22:06:39 -07:00
Gabe Kangas
4f73da30a8 Add additional stream + server info to stream started webhook. Closes #971 2021-04-29 17:04:33 -07:00
Gabe Kangas
eedc07fc94 Ignore warning about hls segment length 2021-04-24 14:00:35 -07:00
Gabe Kangas
34c4350597 Add some more va-api codec error mappings 2021-04-21 17:48:39 -07:00
Gabe Kangas
a06a9c58ea Add i965-va-driver-shader missing error 2021-04-15 22:25:28 -07:00
Gabe Kangas
543577c2e0 Create hls directories at transcoder start to account for stream output changes. Fixes #940 2021-04-15 21:36:14 -07:00
Gabe Kangas
5214d81264
Codec selection (#892)
* Query for installed codecs

* Start modeling out codecs

* Can now specify a codec and get the correct settings returned from the model

* Return codecs in admin/serverconfig

* Start handling transcoding errors and return messages to user

* filter available codecs against a whitelist

* Fix merge

* Codecs are working

* Switching between codecs work

* Add apis for setting a custom video codec

* Cleanup the logging of transcoder errors

* Add v4l codec

* Add fetching v4l

* Add support for per-codec presets

* Use updated nvenc encoding parameters

* Update log message

* Some more codec WIP

* Turn off v4l. It is a mess.

* Try to make the lowest latency level a bit more playable

* Use a human redable display name in console messages

* Turn on transcoder persistent connections

* Add more codec-related user-facing error messages

* Give the initial offline state transcoder an id

* Force a minimum segment count of 3

* Disable qsv for now. set x264 specific params in VariantFlags

* Close body in case

* Ignore vbv underflow message, it is not actionable

* Determine a dynamic gop value based on the length of segments

* Add codec-specific tests

* Cleanup

* Ignore goconst lint warnings in codec file

* Troubleshoot omx

* Add more codec tests

* Remove no longer accurate comment

* Bundle admin from codec branch

* Revert back to old setting

* Cleanup list of codecs a bit

* Remove old references to the encoder preset

* Commit updated API documentation

* Update admin bundle

* Commit updated API documentation

* Add codec setting to api spec

* Commit updated API documentation

Co-authored-by: Owncast <owncast@owncast.online>
2021-04-15 13:55:51 -07:00
Gabe Kangas
f216e781dd Reduce log level on data getters 2021-04-14 16:06:22 -07:00
Gabe Kangas
df7dff081f Set and get custom styling for web interface. For #718 2021-04-11 19:34:05 -07:00
Gabe Kangas
36be7b76c2
If no logo exists copy a default one into data on launch. Closes #909 (#916) 2021-04-11 16:44:50 -07:00
Gabe Kangas
12509c1bc2 Change log level 2021-04-07 00:13:30 -07:00
Gabe Kangas
64a5720a21 Allow unique streampipe fifos. Closes #883 2021-04-01 18:56:00 -07:00
Gabe Kangas
7556671afc Fix race condition where initial offline video state was not written 2021-03-31 23:08:27 -07:00
Gabe Kangas
3ad221665e Handle un-typed or un-parsable chat messages by throwing them away and not panicing. Closes #856 2021-03-25 19:20:05 -07:00
nebunez
80579c5e7e
implement custom welcome message (#820)
* implement custom welcome message

This change adds logic for handling custom welcome messages.

* trim welcome message string on POST

Instead of trimming the welcome message every time we are going to send to send
a welcome message, we just trim once when the message is posted from the admin.
2021-03-21 17:10:56 -04:00
Gabe Kangas
3fb80554ef
Gek/external actions (#827)
* WIP External actions modal frontend

* Add external action links

* Allow modal to show/hide and use a dynamic url

* Use external link object instead of just url for state

* add style and placement to external action buttons

* reformat and simplify tag list style as not to conflict with action buttons and make them look less actionable since they're not

* fix bug to open modal

* have Esc key close modal

* fix style on modal

* make modal bg darker

* close modal when you click outside of it

* fix zindex

* Add support for external action icons and colors

* Some external action modal sizing + loading spinner

Co-authored-by: Ginger Wong <omqmail@gmail.com>
2021-03-15 15:32:52 -07:00
Gabe Kangas
bf33d08384
Add support for disabling chat. Closes #472 (#799) 2021-03-14 11:46:27 -07:00
Gabe Kangas
6f545a905b
Support full html in system messages. Closes #747 (#814) 2021-03-12 00:43:10 -08:00
nebunez
7e0822b62d fix mismatched message ids when client first sends a message
This bug is caused by the message being sent to all clients before
msg.SetDefaults() is called and the message is saved to the chat database. This
causes connected clients to get a message id that is different from what exists
in the database. This only affects clients using websockets that were connected
when the message was sent and message visibility updated.

Fixes https://github.com/owncast/owncast/issues/809
2021-03-10 14:07:47 -05:00
Gabe Kangas
de195f883e
Gek/disable remote images (#800)
* Disable images from anywhere but our emojis. Closes #756

* Add tests around images in chat messages

* Update sanitizer + test
2021-03-08 23:20:15 -08:00
Gabe Kangas
a67e938875 allocate yp at launch to allow for enabling it later 2021-03-04 01:48:10 -08:00
Gabe Kangas
49bb899d51 Optimize/fix concurrency for chat 2021-03-03 21:48:11 -08:00
Gabe Kangas
d8e6b75b5f No longer reset max viewer count at cleanup. Only do it at new stream start. Fixes #768 2021-02-26 22:15:46 -08:00
Bradley Hilton
cbcf8d4794
Fix the http port used not in sync with what is logged (#766)
* Fix the http port used not in sync with what is logged

* Actually fix the logging race condition

* Move the router logging to the router package
2021-02-26 11:23:15 -08:00
Gabe Kangas
4f833be80a Add ability to mark a message as ephemeral and set it with title changes 2021-02-23 11:24:32 -08:00
Gabe Kangas
0bfd9f8e1f Do not send user joined events to chat clients if stream is not active. Closes #750 2021-02-21 16:21:12 -08:00
Gabe Kangas
b47589fa8f Print out a log message saying what image is being copied where 2021-02-21 16:03:42 -08:00
Gabe Kangas
ebc3070ea0 Assume videopassthrough is enabled if bitrate is zero 2021-02-21 13:10:24 -08:00
Gabe Kangas
b1368db079 Fix yaml property name 2021-02-21 13:06:32 -08:00
Gabe Kangas
765dd7efe2 Fix chat history backlog query. Closes #748 2021-02-21 12:13:46 -08:00
Gabe Kangas
bc2caadb74
0.0.6 -> Master (#731)
* Implement webhook events for external integrations (#574)

* Implement webhook events for external integrations

Reference #556

* move message type to models and remove duplicate

* add json header so content type can be determined

* Pass at migrating webhooks to datastore + management apis (#589)

* Pass at migrating webhooks to datastore + management apis

* Support nil lastUsed timestamps and return back the new webhook on create

* Cleanup from review feedback

* Simplify a bit

Co-authored-by: Aaron Ogle <aaron@geekgonecrazy.com>

Co-authored-by: Gabe Kangas <gabek@real-ity.com>

* Webhook query cleanup

* Access tokens + Send system message external API (#585)

* New add, get and delete access token APIs

* Create auth token middleware

* Update last_used timestamp when using an access token

* Add auth'ed endpoint for sending system messages

* Cleanup

* Update api spec for new apis

* Commit updated API documentation

* Add auth'ed endpoint for sending user chat messages

* Return access token string

* Commit updated API documentation

* Fix route

* Support nil lastUsed time

* Commit updated Javascript packages

* Remove duplicate function post rebase

* Fix msg id generation

* Update controllers/admin/chat.go

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Webhook query cleanup

* Add SystemMessageSent to EventType

Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>

* Set webhook as used on completion. Closes #610

* Display webhook errors as errors

* Commit updated API documentation

* Add user joined chat event

* Change integration API paths. Update API spec

* Update development version of admin that supports integration apis

* Commit updated API documentation

* Add automated tests for external integration APIs

* check error

* quiet this test for now

* Route up some additional 3rd party apis. #638

* Commit updated API documentation

* Save username on user joined event

* Add missing scope to valid scopes list

* Add generic chat action event API for 3rd parties. Closes #666

* Commit updated API documentation

* First pass at moving WIP config framework into project for #234

* Only support exported fields in custom types

* Using YP get/set key as a first pass at using the data layer. Fixes + integration.

* Ignore test db

* Start adding getters and setters for config values

* More get/set config work. Starting to populate api with data

* Wire up some config edit endpoints

* More endpoints

* Disable cors middleware

* Add more endpoints and add test to test them

* Remove the in-memory change APIs

* Add endpoint for changing tags

* Add more config endpoints

* Starting to point more things away from config file and to the datastore

* Populate YP with db data

* Create new util method for parsing page body markdown and return it in api

* Verify proposed path to ffmpeg

* For development purposes show the config key in logs

* Move stats values to datastore

* Moving over more values to the datastore

* Move S3 config to datastore

* First pass the config -> db migrator

* Add the start of the video config apis

* It builds pointing everything away from the config

* Tweak ffmpeg path error message

* Backup database every hour. Closes #549

* Config + defaults + migration work for db

* Cleanup logging

* Remove all the old config structs

* Add descriptive info about migration

* Tweak ffmpeg validation logic

* Fix db backup path. backup on db version migration

* Set video and s3 configurations

* Update api spec with new config endpoints

* Add migrator for stats file

* Commit updated API documentation

* Use a dynamic system port for internal HLS writes. Closes #577 (#626)

* Use a dynamic system port for internal HLS writes. Closes #577

* Cleanup

* YP key migration to datastore

* Create a backup directory if needed before migrations

* Remove config test that no longer makes sense. Cleanup.

* Change number types from float32 to float64

* Update automated test suite

* Allow restoring a database backup via command line flags. Closes #549

* Add new hls segment config api

* Commit updated API documentation

* Update apis to require a value container property

* add socialHandles api

* Commit updated API documentation

* Add new latancy level setting to replace segment settings

* Commit updated API documentation

* Fix spelling

* Commit updated API documentation

* hardcode a json api of available social platforms

* Add additional icons

* Return social handles in server config api

* Add socialhandles validation to test

* Move list of hard coded social platforms to an api

* Remove audio only code from transcoder since we do not use it

* Add latency levels api + snapshot of video settings as current broadcast

* Add config/serverurl endpoint

* Return 404 on YP api if disabled

* Surface stream title in YP response

* Add stream title to web ui

* Cleanup log message. Closes #520

* Rename ffmpeg package to transcoder

* Add ws package for testing

* Reduce chat backlog to past 5hrs, max 50. Closes #548

* Fix error formatting

* Add endpoint for resetting yp registration

* Add yp/reset to api spec. return status in response

* Return zero viewer count if stream is offline. Closes #422

* Post-rebase fixes

* Fix merge conflict in openapi file

* Commit updated API documentation

* Standardize controller names

* Support setting the stream key via the command line. Closes #665

* Return social handles with YP data. First half of https://github.com/owncast/owncast-yp/issues/28

* Give the YP package access to server status regardless if enabled or not

* Change delay in automated tests

* Add stream title integration API. For #638

* Commit updated API documentation

* Add storage to the migrator

* Missing returning NSFW value in server config

* Add flag to ignore websocket client. Closes #537

* Add error for parsing broadcaster metadata

* Add support for a cli specified http server port. Closes #674

* Add cpu usage levels and a temporary mapping between it and libx264 presets

* Test for valid url endpoint when saving s3 config

* Re-configure storage on every stream to allow changing storage providers

* After 5 minutes of a stream being stopped clear the stream title

* Hide viewer count once stream goes offline instead of when player stops

* Pull steamTitle from the status that gets updated instead of the config

* Commit updated API documentation

* Optionally show stream title in the header

* Reset stream title when server starts

* Show chat action when stream title is updated

* Allow system messages to come back in persistence

* Split out getting chat history for moderation + fix tests

* Remove server title and standardize on name only

* Commit updated API documentation

* Bump github.com/aws/aws-sdk-go from 1.37.1 to 1.37.2 (#680)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.1 to 1.37.2.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.1...v1.37.2)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Add video variant and stream latency config file migrator

* Remove mostly unused disable upgrade check bool

* Commit updated API documentation

* Allow bundling the admin from the 0.0.6 branch

* Fix saving port numbers

* Use name instead of old title on window focus

* Work on latency levels. Fix test to use levels. Clean up transcoder to only reference levels

* Another place where title -> name

* Fix test

* Bump github.com/aws/aws-sdk-go from 1.37.2 to 1.37.3 (#690)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.2 to 1.37.3.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.2...v1.37.3)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Update dependabot config

* Bump github.com/aws/aws-sdk-go from 1.37.3 to 1.37.5 (#693)

Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.37.3 to 1.37.5.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Changelog](https://github.com/aws/aws-sdk-go/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.37.3...v1.37.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript (#694)

* Bump video.js from 7.10.2 to 7.11.4 in /build/javascript

Bumps [video.js](https://github.com/videojs/video.js) from 7.10.2 to 7.11.4.
- [Release notes](https://github.com/videojs/video.js/releases)
- [Changelog](https://github.com/videojs/video.js/blob/main/CHANGELOG.md)
- [Commits](https://github.com/videojs/video.js/compare/v7.10.2...v7.11.4)

Signed-off-by: dependabot[bot] <support@github.com>

* Commit updated Javascript packages

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>

* Make the latency migrator dynamic so I can tweak values easier

* Split out fetching ffmpeg path from validating the path so it can be changed in the admin

* Some commenting and linter cleanup

* Validate the path for a logo change and throw an error if it does not exist

* Logo change requests have to be a real file now

* Cleanup, making linter happy

* Format javascript on push

* Only format js in master

* Tweak latency level values

* Remove unused config file examples

* Fix thumbnail generation after messing with the ffmpeg path getter

* Reduce how often we report high hardware utilization warnings

* Bundle the 0.0.6 branch version of the admin

* Return validated ffmpeg path in admin server config

* Change the logo to be stored in the data directory instead of webroot

* Bump postcss from 8.2.4 to 8.2.5 in /build/javascript (#702)

Bumps [postcss](https://github.com/postcss/postcss) from 8.2.4 to 8.2.5.
- [Release notes](https://github.com/postcss/postcss/releases)
- [Changelog](https://github.com/postcss/postcss/blob/main/CHANGELOG.md)
- [Commits](https://github.com/postcss/postcss/compare/8.2.4...8.2.5)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Default config file no longer used

* don't show stream title when offline

addresses https://github.com/owncast/owncast/issues/677

* Remove auto-clearing stream title. #677

* webroot -> data when using logo as thumbnail

* Do not list websocket/access token create/delete as integration APIs

* Commit updated API documentation

* Bundle updated admin

* Remove pointing to the 0.0.6 admin branch

* Linter cleanup

* Linter cleanup

* Add donations and follow links to show up under social handles

* Prettified Code!

* More linter cleanup

* Update admin bundle

* Remove use of platforms.js and return icons with social handles. Closes #732

* Update admin bundle

* Support custom config path for use in migration

* Remove unused platform-logos.gif

* Reduce log level of message

* Remove unused logo files in static dir

* Handle dev vs. release build info

* Restore logo.png for initial thumbnail

* Cleanup some files from the build process that are not needed

* Fix incorrect build-time injection var

* Fix missing file getting copied to the build

* Remove console directory message.

* Update admin bundle

* Fix comment

* Report storage setup error

* add some value set error checking

* Use validated dynamic ffmpeg path for animated gif preview

* Make chat message links be white so they don't hide in the bg. Closes #599

* Restore conditional that was accidentally removed

Co-authored-by: Aaron Ogle <geekgonecrazy@users.noreply.github.com>
Co-authored-by: Owncast <owncast@owncast.online>
Co-authored-by: Ginger Wong <omqmail@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: nebunez <uoj2y7wak869@opayq.net>
Co-authored-by: gabek <gabek@users.noreply.github.com>
2021-02-18 23:05:52 -08:00
Fabian Siegel
4bfdf0e667
Fix codeblocks in chat (#676)
* allowed codeblocks in chat (owncast#675)

* dont send empty chat messages back to frontend (owncast#675)
2021-01-31 14:57:50 -08:00
Gabe Kangas
89cf6115b3 Give more useful error messages and don't panic on bad disconnects. Closes #673 2021-01-29 11:25:18 -08:00
Gabe Kangas
cf54be4b1d Stop logging rate limit messages 2021-01-26 19:52:32 -08:00
petersveter108
401995c52e
Websocket client errors as Warns, broken pipe errors as Trace (#656)
* Websocket client errors now logged as Warns
broken pipe errors now logged as Trace

* additional space for broken pipe logs
2021-01-22 13:50:23 -08:00
Gabe Kangas
eacd0bcbd4 check error from read timeout 2021-01-16 17:24:12 -08:00
Gabe Kangas
2b81fcef53 Add a read timeout when reading rtmp to handle connections that have dropped. Closes #564 2021-01-13 21:58:34 -08:00
Gabe Kangas
acd546ce6c Remove fatal error. Closes #623 2021-01-13 17:57:37 -08:00
Gabe Kangas
bda430f791 Cleanup some exported methods 2021-01-01 17:08:20 -08:00
Gabe Kangas
1dc25889ec Add inbound chat event rate limiting. Closes #484 2021-01-01 16:55:46 -08:00
Gabe Kangas
b2953028cf Remove forcing old path-style URLs with s3. Closes #497 2020-12-31 11:50:18 -08:00
Gabe Kangas
bc4b6895be Fix error message 2020-12-30 12:22:44 -08:00
gingervitis
8a74af202d
Update chat message visibility for moderation (#524)
* update message viz in db

* create admin endpoint to update message visibility

* convert UpdateMessageVisibility api to take in an array of IDs to change visibility on instead

* Support requesting filtered or unfiltered chat messages

* Handle UPDATE chat events on front and backend for toggling messages

* Return entire message with UPDATE events

* Remove the UPDATE message type

* Revert "Remove the UPDATE message type"

This reverts commit 3a83df3d492f7ecf2bab65e845aa2b0365d3a7f6.

* update -> visibility update

* completely remove messages when they turn hidden on VISIBILITY-UPDATEs, and insert them if they turn visible

* Explicitly set visibility

* Fix multi-id sql updates

* increate scroll buffer a bit so chat scrolls when new large messages come in

* Add automated test around chat moderation

* Add new chat admin APIs to api spec

* Commit updated API documentation

Co-authored-by: Gabe Kangas <gabek@real-ity.com>
Co-authored-by: Owncast <owncast@owncast.online>
2020-12-29 13:35:33 -08:00
Gabe Kangas
e558c549d7
Websocket fixes (#461)
* Bump api spec version

* Do not panic on cpu monitoring error

* Centralize the socket disconnect logic and fire it also when socket errors occur. Hopefully closes #421
2020-12-21 19:42:47 -08:00
Christian Muehlhaeuser
c7cc973ebc
Log and abort when listening on RTMP port fails (#478)
Fixes #477.
2020-12-19 22:17:10 -08:00
Gabe Kangas
e9c7ff29cf
Add custom RTMP port number support (#462)
* Bump api spec version

* Add support for custom RTMP port. Closes #454
2020-12-17 09:56:04 -08:00
Gabe Kangas
1fe1c6c925 Move duplicate string to a constant 2020-12-16 23:01:09 -08:00
Gabe Kangas
47770feaa5 Remove TODO 2020-12-16 22:59:13 -08:00
Gabe Kangas
b405b2fcc7 Connections from Restream seem to be more sensitive about if we get the initial packets processed or not, so I moved setStreamAsConnected to be decoupled from when we set broadcaster info. Another update for #432 2020-12-06 14:28:00 -08:00
Gabe Kangas
1c42bf9f63 Check for nil when accessing broadcaster. Might help #432 2020-12-06 12:01:16 -08:00
Gabe Kangas
2abea78e3c Use onMetaData instead of setDataFrame to try to parse out RTMP
metadata. Closes #419
2020-12-05 18:37:42 -08:00
Gabe Kangas
4a4639a523 Ignore old per-session peak viewer counts when reading stats file. 2020-12-05 15:09:57 -08:00
Gabe Kangas
0b66f3b79f
Support video-only streams if there is no audio in the inbound stream. (#401)
Closes #400
2020-12-02 00:19:55 -08:00
Gabe Kangas
4cb0027ed3 Fix blocking YP 2020-11-20 22:59:20 -08:00