mirror of
https://github.com/owncast/owncast.git
synced 2024-11-25 22:31:09 +03:00
b6f68628c0
* First pass at chat user registration and validation * Disable chat if the user is disabled/blocked or the server hits max connections * Handle dropping sockets if chat is disabled * Fix origin in automated chat test * Work for updated chat moderation * Chat message markdown rendering and fix tests * Put /api/chat behind a chat user access token. Closes #1085 * Reject blocked username changes * More WIP moderation * Defer configuring chat until we know if it is enabled. Closes #1135 * chat user blocking. Closes #1096 * Add tests around user access for #1096 * Add external integration chat message API + update integration auth middleware to pass along integration name. Closes #1092 * Delete old chat messages from db as to not hold on to excessive data. Closes #1152 * Add schema migration for messages. Closes #1155 * Commit updated API documentation * Add chat load test * Shared db mutex and db optimizations * Simplify past display name handling * Use a new test db for each test run * Wire up the external messages actions + add tests for them * Move access tokens to be actual users * Run message pruning at launch + fix comparison * Do not return API users in disabled users response * Fix incorrect highlighting. Closes #1160 * Consolidate user table statements * Set the max process connection limit to 70% of maximum * Fix wrong old display name being returned in name change event * Delete the old chat server files * Wire back up the webhooks * Remove unused * Invalidate user cache on changes * Do not send rendered body as RawBody * Some cleanup * Standardize names for external API users to ExternalAPIUser * Do not log token * Checkout branch when building admin for testing * Bundle in dev admin for testing * Some cleanup * Cleanup js logs * Cleanup and standardize event names * Clean up some logging * Update API spec. Closes #1133 * Commit updated API documentation * Change paths to be better named * Commit updated API documentation * Update admin bundle * Fix duplicate event name * Rename scope var * Update admin bundle * Move connected clients controller into admin package * Fix collecting usernames for autocomplete purposes * No longer generate username when it is empty * Sort clients and users by timestamp * Move file to admin controller package * Swap, so the comments stay correct Co-authored-by: Jannik <jannik@outlook.com> * Use explicit type alias Co-authored-by: Jannik <jannik@outlook.com> * Remove commented code. Co-authored-by: Jannik <jannik@outlook.com> * Cleanup test * Remove some extra logging * Add some clarity * Update dev instance of admin for testing * Consolidate lines Co-authored-by: Jannik <jannik@outlook.com> * Remove commented unused vars Co-authored-by: Jannik <jannik@outlook.com> * Until needed do not return IP address with client list * Fix typo of wrong var * Typo led to a bad test. Fix typo and fix test. * Guard against the socket reconnecting on error if previously set to shutdown * Do not log access tokens * Return success message on enable/disable user * Clean up some inactionable error messages. Sent ban message. Sort banned users. * fix styling for when chat is completely disabled * Unused * guard against nil clients * Update dev admin bundle * Do not unhide messages when unblocking user just to be safe. Send removal action from the controller * Add convinience function for getting active connections for a single user * Lock db on these mutations * Cleanup force disconnect using GetClientsForUser and capture client reference explicitly * No longer re-showing banned user messages for safety. Removing this test. * Remove no longer needed comment * Tweaks to forbidden username handling. - Standardize naming to not use "block" but "forbidden" instead. - Pass array over the wire instead of string. - Add API test - Fix default list incorrectly being appended to custom list. * Logging cleanup * Update dev admin bundle * Add an artificial delay in order to visually see message being hidden when testing * Remove the user cache as it is a premature optimization * When connected to chat let the user know their current user details to sync the username in the UI * On connected send current display name back to client. - Move name change out of chat component. - Add additional event type constants. * Fix broken workflow due to typo * Troubleshoot workflow * Bump htm from 3.0.4 to 3.1.0 in /build/javascript (#1181) * Bump htm from 3.0.4 to 3.1.0 in /build/javascript Bumps [htm](https://github.com/developit/htm) from 3.0.4 to 3.1.0. - [Release notes](https://github.com/developit/htm/releases) - [Commits](https://github.com/developit/htm/compare/3.0.4...3.1.0) --- updated-dependencies: - dependency-name: htm dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> * Run npm run build and update libraries Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Gabe Kangas <gabek@real-ity.com> * Commit updated Javascript packages * Re-send current user info when a rejected name change takes place * All socket writes should be through the send chan and not directly * Seed the random generator * Add keys and indexes to users table * a util to generate consistent emoji markup * console clean up * mod tidy * Commit updated API documentation * Handle the max payload size of a socket message. - Only close socket if x2 greater than the max size. - Send the user a message if a message is too large. - Surface the max size in bytes in the config. * Update admin bundle * Force all events to be sent in their own socket message and do not concatinate in a single message * Update chat embed to register for access token * Use a different access token for embed chat * Update the chat message bubble background color to be bolder * add base tag to open links in new window, closes #1220 * Support text input of :emoji: in chat (#1190) * Initial implementation of emoji injection * fix bookkeeping with multiple emoji * make the emoji lookup case-insensitive * try another solution for Caretposition * add title to emojis minor refactoring * bind moji injection to InputKeyUp * simplify the code replace all found emojis * inject emoji if the modifer is released earlier * more efficient emoji tag search * use json emoji.emoji as url * use createEmojiMarkup() * move emojify() to chat.js * emojify on paste * cleanup emoji titles in paste * update inputText in InputKeyup * mark emoji titles with 2*zwnj this way paste cleanup will not interfere with text which include zwnj * emoji should not change the inputText * Do not show join messages when chat is offline. Closes #1224 - Show stream starting/ending messages in chat. - When stream starts show everyone the welcome message. * Force scrolling chat to bottom after history is populated regardless of scroll position. Closes https://github.com/owncast/owncast/issues/1222 * use maxSocketPayloadSize to calculate total bytes of message payload (#1221) * utilize maxSocketPayloadSize from config; update chatInput to calculate based on that value instead of text value; remove usage of inputText for counting * add a buffer to account for entire websocket payload for message char counting; trim nbsp;'s from ends of messages when calculating count Co-authored-by: Gabe Kangas <gabek@real-ity.com> Co-authored-by: Owncast <owncast@owncast.online> Co-authored-by: Jannik <jannik@outlook.com> Co-authored-by: Ginger Wong <omqmail@gmail.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Meisam <39205857+MFTabriz@users.noreply.github.com>
257 lines
6.1 KiB
Go
257 lines
6.1 KiB
Go
package utils
|
|
|
|
import (
|
|
"bytes"
|
|
"errors"
|
|
"fmt"
|
|
"io/ioutil"
|
|
"math/rand"
|
|
"net/url"
|
|
"os"
|
|
"os/exec"
|
|
"path"
|
|
"path/filepath"
|
|
"strings"
|
|
|
|
"github.com/mssola/user_agent"
|
|
log "github.com/sirupsen/logrus"
|
|
"github.com/yuin/goldmark"
|
|
"github.com/yuin/goldmark/extension"
|
|
"github.com/yuin/goldmark/renderer/html"
|
|
"mvdan.cc/xurls"
|
|
)
|
|
|
|
// DoesFileExists checks if the file exists.
|
|
func DoesFileExists(name string) bool {
|
|
if _, err := os.Stat(name); err == nil {
|
|
return true
|
|
} else if os.IsNotExist(err) {
|
|
return false
|
|
} else {
|
|
log.Errorln(err)
|
|
return false
|
|
}
|
|
}
|
|
|
|
// GetRelativePathFromAbsolutePath gets the relative path from the provided absolute path.
|
|
func GetRelativePathFromAbsolutePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
file := pathComponents[len(pathComponents)-1]
|
|
|
|
return filepath.Join(variant, file)
|
|
}
|
|
|
|
func GetIndexFromFilePath(path string) string {
|
|
pathComponents := strings.Split(path, "/")
|
|
variant := pathComponents[len(pathComponents)-2]
|
|
|
|
return variant
|
|
}
|
|
|
|
// Copy copies the file to destination.
|
|
func Copy(source, destination string) error {
|
|
input, err := ioutil.ReadFile(source)
|
|
if err != nil {
|
|
return err
|
|
}
|
|
|
|
return ioutil.WriteFile(destination, input, 0600)
|
|
}
|
|
|
|
// Move moves the file to destination.
|
|
func Move(source, destination string) error {
|
|
return os.Rename(source, destination)
|
|
}
|
|
|
|
// IsUserAgentABot returns if a web client user-agent is seen as a bot.
|
|
func IsUserAgentABot(userAgent string) bool {
|
|
if userAgent == "" {
|
|
return false
|
|
}
|
|
|
|
botStrings := []string{
|
|
"mastodon",
|
|
"pleroma",
|
|
"applebot",
|
|
}
|
|
|
|
for _, botString := range botStrings {
|
|
if strings.Contains(strings.ToLower(userAgent), botString) {
|
|
return true
|
|
}
|
|
}
|
|
|
|
ua := user_agent.New(userAgent)
|
|
return ua.Bot()
|
|
}
|
|
|
|
func RenderSimpleMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
return buf.String()
|
|
}
|
|
|
|
func RenderPageContentMarkdown(raw string) string {
|
|
markdown := goldmark.New(
|
|
goldmark.WithRendererOptions(
|
|
html.WithUnsafe(),
|
|
),
|
|
goldmark.WithExtensions(
|
|
extension.GFM,
|
|
extension.NewLinkify(
|
|
extension.WithLinkifyAllowedProtocols([][]byte{
|
|
[]byte("http:"),
|
|
[]byte("https:"),
|
|
}),
|
|
extension.WithLinkifyURLRegexp(
|
|
xurls.Strict,
|
|
),
|
|
),
|
|
),
|
|
)
|
|
|
|
trimmed := strings.TrimSpace(raw)
|
|
var buf bytes.Buffer
|
|
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
|
|
panic(err)
|
|
}
|
|
|
|
return strings.TrimSpace(buf.String())
|
|
}
|
|
|
|
// GetCacheDurationSecondsForPath will return the number of seconds to cache an item.
|
|
func GetCacheDurationSecondsForPath(filePath string) int {
|
|
if path.Base(filePath) == "thumbnail.jpg" {
|
|
// Thumbnails re-generate during live
|
|
return 20
|
|
} else if path.Ext(filePath) == ".js" || path.Ext(filePath) == ".css" {
|
|
// Cache javascript & CSS
|
|
return 60
|
|
} else if path.Ext(filePath) == ".ts" {
|
|
// Cache video segments as long as you want. They can't change.
|
|
// This matters most for local hosting of segments for recordings
|
|
// and not for live or 3rd party storage.
|
|
return 31557600
|
|
} else if path.Ext(filePath) == ".m3u8" {
|
|
return 0
|
|
}
|
|
|
|
// Default cache length in seconds
|
|
return 30
|
|
}
|
|
|
|
func IsValidUrl(urlToTest string) bool {
|
|
if _, err := url.ParseRequestURI(urlToTest); err != nil {
|
|
return false
|
|
}
|
|
|
|
u, err := url.Parse(urlToTest)
|
|
if err != nil || u.Scheme == "" || u.Host == "" {
|
|
return false
|
|
}
|
|
|
|
return true
|
|
}
|
|
|
|
// ValidatedFfmpegPath will take a proposed path to ffmpeg and return a validated path.
|
|
func ValidatedFfmpegPath(ffmpegPath string) string {
|
|
if ffmpegPath != "" {
|
|
if err := VerifyFFMpegPath(ffmpegPath); err == nil {
|
|
return ffmpegPath
|
|
} else {
|
|
log.Warnln(ffmpegPath, "is an invalid path to ffmpeg will try to use a copy in your path, if possible")
|
|
}
|
|
}
|
|
|
|
// First look to see if ffmpeg is in the current working directory
|
|
localCopy := "./ffmpeg"
|
|
hasLocalCopyError := VerifyFFMpegPath(localCopy)
|
|
if hasLocalCopyError == nil {
|
|
// No error, so all is good. Use the local copy.
|
|
return localCopy
|
|
}
|
|
|
|
cmd := exec.Command("which", "ffmpeg")
|
|
out, err := cmd.CombinedOutput()
|
|
if err != nil {
|
|
log.Fatalln("Unable to determine path to ffmpeg. Please specify it in the config file.")
|
|
}
|
|
|
|
path := strings.TrimSpace(string(out))
|
|
return path
|
|
}
|
|
|
|
// VerifyFFMpegPath verifies that the path exists, is a file, and is executable.
|
|
func VerifyFFMpegPath(path string) error {
|
|
stat, err := os.Stat(path)
|
|
|
|
if os.IsNotExist(err) {
|
|
return errors.New("ffmpeg path does not exist")
|
|
}
|
|
|
|
if err != nil {
|
|
return fmt.Errorf("error while verifying the ffmpeg path: %s", err.Error())
|
|
}
|
|
|
|
if stat.IsDir() {
|
|
return errors.New("ffmpeg path can not be a folder")
|
|
}
|
|
|
|
mode := stat.Mode()
|
|
//source: https://stackoverflow.com/a/60128480
|
|
if mode&0111 == 0 {
|
|
return errors.New("ffmpeg path is not executable")
|
|
}
|
|
|
|
return nil
|
|
}
|
|
|
|
// Removes the directory and makes it again. Throws fatal error on failure.
|
|
func CleanupDirectory(path string) {
|
|
log.Traceln("Cleaning", path)
|
|
if err := os.RemoveAll(path); err != nil {
|
|
log.Fatalln("Unable to remove directory. Please check the ownership and permissions", err)
|
|
}
|
|
if err := os.MkdirAll(path, 0777); err != nil {
|
|
log.Fatalln("Unable to create directory. Please check the ownership and permissions", err)
|
|
}
|
|
}
|
|
|
|
func FindInSlice(slice []string, val string) (int, bool) {
|
|
for i, item := range slice {
|
|
if item == val {
|
|
return i, true
|
|
}
|
|
}
|
|
return -1, false
|
|
}
|
|
|
|
// GenerateRandomDisplayColor will return a random _hue_ to be used when displaying a user.
|
|
// The UI should determine the right saturation and lightness in order to make it look right.
|
|
func GenerateRandomDisplayColor() int {
|
|
rangeLower := 0
|
|
rangeUpper := 360
|
|
return rangeLower + rand.Intn(rangeUpper-rangeLower+1) //nolint
|
|
}
|