owncast/utils/utils.go

134 lines
3 KiB
Go
Raw Normal View History

package utils
import (
"bytes"
"io/ioutil"
"os"
HLS video handling/storage/state refactor (#151) * WIP with new transcoder progress monitor * A whole different WIP in progress monitoring via local PUTs * Use an actual hls playlist parser to rewrite master playlist * Cleanup * Private vs public path for thumbnail generation * Allow each storage provider to make decisions of how to store different types of files * Simplify inbound file writes * Revert * Split out set stream as connected/disconnected state methods * Update videojs * Add comment about the hls handler * Rework of the offline stream state. For #85 * Delete old unreferenced video segment files from disk * Cleanup all segments and revert to a completely offline state after 5min * Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup. * Update transcoder test * Add comment * Return http 200 on success to transcoder. Tweak how files are written to disk * Force pixel color format in transcoder * Add debugging info for S3 transfers. Add default ACL. * Fix cleanup timer * Reset session stats when we cleanup the session. * Put log file back * Update test * File should not be a part of this commit * Add centralized shared performance timer for use anywhere * Post-rebase cleanup * Support returning nil from storage provider save * Updates to reflect package changes + other updates in master * Fix storage providers being overwritten * Do not return pointer in save. Support cache headers with S3 providers * Split out videojs + vhs and point to specific working versions of them * Bump vjs and vhs versions * Fix test * Remove unused * Update upload warning message * No longer valid comment * Pin videojs and vhs versions
2020-10-15 00:07:38 +03:00
"path"
"path/filepath"
"strings"
"github.com/mssola/user_agent"
"github.com/yuin/goldmark"
"github.com/yuin/goldmark/extension"
"github.com/yuin/goldmark/renderer/html"
"mvdan.cc/xurls"
)
2020-11-13 02:14:59 +03:00
// GetTemporaryPipePath gets the temporary path for the streampipe.flv file.
func GetTemporaryPipePath() string {
2020-07-12 02:04:02 +03:00
return filepath.Join(os.TempDir(), "streampipe.flv")
}
2020-11-13 02:14:59 +03:00
// DoesFileExists checks if the file exists.
func DoesFileExists(name string) bool {
if _, err := os.Stat(name); err != nil {
if os.IsNotExist(err) {
return false
}
}
return true
}
2020-11-13 02:14:59 +03:00
// GetRelativePathFromAbsolutePath gets the relative path from the provided absolute path.
func GetRelativePathFromAbsolutePath(path string) string {
pathComponents := strings.Split(path, "/")
variant := pathComponents[len(pathComponents)-2]
file := pathComponents[len(pathComponents)-1]
return filepath.Join(variant, file)
}
HLS video handling/storage/state refactor (#151) * WIP with new transcoder progress monitor * A whole different WIP in progress monitoring via local PUTs * Use an actual hls playlist parser to rewrite master playlist * Cleanup * Private vs public path for thumbnail generation * Allow each storage provider to make decisions of how to store different types of files * Simplify inbound file writes * Revert * Split out set stream as connected/disconnected state methods * Update videojs * Add comment about the hls handler * Rework of the offline stream state. For #85 * Delete old unreferenced video segment files from disk * Cleanup all segments and revert to a completely offline state after 5min * Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup. * Update transcoder test * Add comment * Return http 200 on success to transcoder. Tweak how files are written to disk * Force pixel color format in transcoder * Add debugging info for S3 transfers. Add default ACL. * Fix cleanup timer * Reset session stats when we cleanup the session. * Put log file back * Update test * File should not be a part of this commit * Add centralized shared performance timer for use anywhere * Post-rebase cleanup * Support returning nil from storage provider save * Updates to reflect package changes + other updates in master * Fix storage providers being overwritten * Do not return pointer in save. Support cache headers with S3 providers * Split out videojs + vhs and point to specific working versions of them * Bump vjs and vhs versions * Fix test * Remove unused * Update upload warning message * No longer valid comment * Pin videojs and vhs versions
2020-10-15 00:07:38 +03:00
func GetIndexFromFilePath(path string) string {
pathComponents := strings.Split(path, "/")
variant := pathComponents[len(pathComponents)-2]
return variant
}
2020-11-13 02:14:59 +03:00
// Copy copies the file to destination.
func Copy(source, destination string) error {
input, err := ioutil.ReadFile(source)
if err != nil {
return err
}
return ioutil.WriteFile(destination, input, 0600)
}
2020-11-13 02:14:59 +03:00
// Move moves the file to destination.
HLS video handling/storage/state refactor (#151) * WIP with new transcoder progress monitor * A whole different WIP in progress monitoring via local PUTs * Use an actual hls playlist parser to rewrite master playlist * Cleanup * Private vs public path for thumbnail generation * Allow each storage provider to make decisions of how to store different types of files * Simplify inbound file writes * Revert * Split out set stream as connected/disconnected state methods * Update videojs * Add comment about the hls handler * Rework of the offline stream state. For #85 * Delete old unreferenced video segment files from disk * Cleanup all segments and revert to a completely offline state after 5min * Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup. * Update transcoder test * Add comment * Return http 200 on success to transcoder. Tweak how files are written to disk * Force pixel color format in transcoder * Add debugging info for S3 transfers. Add default ACL. * Fix cleanup timer * Reset session stats when we cleanup the session. * Put log file back * Update test * File should not be a part of this commit * Add centralized shared performance timer for use anywhere * Post-rebase cleanup * Support returning nil from storage provider save * Updates to reflect package changes + other updates in master * Fix storage providers being overwritten * Do not return pointer in save. Support cache headers with S3 providers * Split out videojs + vhs and point to specific working versions of them * Bump vjs and vhs versions * Fix test * Remove unused * Update upload warning message * No longer valid comment * Pin videojs and vhs versions
2020-10-15 00:07:38 +03:00
func Move(source, destination string) error {
return os.Rename(source, destination)
}
2020-11-13 02:14:59 +03:00
// IsUserAgentABot returns if a web client user-agent is seen as a bot.
func IsUserAgentABot(userAgent string) bool {
if userAgent == "" {
return false
}
botStrings := []string{
"mastodon",
"pleroma",
"applebot",
}
for _, botString := range botStrings {
if strings.Contains(strings.ToLower(userAgent), botString) {
return true
}
}
ua := user_agent.New(userAgent)
return ua.Bot()
}
func RenderSimpleMarkdown(raw string) string {
markdown := goldmark.New(
goldmark.WithRendererOptions(
html.WithUnsafe(),
),
goldmark.WithExtensions(
extension.NewLinkify(
extension.WithLinkifyAllowedProtocols([][]byte{
[]byte("http:"),
[]byte("https:"),
}),
extension.WithLinkifyURLRegexp(
xurls.Strict,
),
),
),
)
trimmed := strings.TrimSpace(raw)
var buf bytes.Buffer
if err := markdown.Convert([]byte(trimmed), &buf); err != nil {
panic(err)
}
return buf.String()
}
HLS video handling/storage/state refactor (#151) * WIP with new transcoder progress monitor * A whole different WIP in progress monitoring via local PUTs * Use an actual hls playlist parser to rewrite master playlist * Cleanup * Private vs public path for thumbnail generation * Allow each storage provider to make decisions of how to store different types of files * Simplify inbound file writes * Revert * Split out set stream as connected/disconnected state methods * Update videojs * Add comment about the hls handler * Rework of the offline stream state. For #85 * Delete old unreferenced video segment files from disk * Cleanup all segments and revert to a completely offline state after 5min * Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup. * Update transcoder test * Add comment * Return http 200 on success to transcoder. Tweak how files are written to disk * Force pixel color format in transcoder * Add debugging info for S3 transfers. Add default ACL. * Fix cleanup timer * Reset session stats when we cleanup the session. * Put log file back * Update test * File should not be a part of this commit * Add centralized shared performance timer for use anywhere * Post-rebase cleanup * Support returning nil from storage provider save * Updates to reflect package changes + other updates in master * Fix storage providers being overwritten * Do not return pointer in save. Support cache headers with S3 providers * Split out videojs + vhs and point to specific working versions of them * Bump vjs and vhs versions * Fix test * Remove unused * Update upload warning message * No longer valid comment * Pin videojs and vhs versions
2020-10-15 00:07:38 +03:00
2020-11-13 02:14:59 +03:00
// GetCacheDurationSecondsForPath will return the number of seconds to cache an item.
HLS video handling/storage/state refactor (#151) * WIP with new transcoder progress monitor * A whole different WIP in progress monitoring via local PUTs * Use an actual hls playlist parser to rewrite master playlist * Cleanup * Private vs public path for thumbnail generation * Allow each storage provider to make decisions of how to store different types of files * Simplify inbound file writes * Revert * Split out set stream as connected/disconnected state methods * Update videojs * Add comment about the hls handler * Rework of the offline stream state. For #85 * Delete old unreferenced video segment files from disk * Cleanup all segments and revert to a completely offline state after 5min * Stop thumbnail generation on stream stop. Copy logo to thumbnail on cleanup. * Update transcoder test * Add comment * Return http 200 on success to transcoder. Tweak how files are written to disk * Force pixel color format in transcoder * Add debugging info for S3 transfers. Add default ACL. * Fix cleanup timer * Reset session stats when we cleanup the session. * Put log file back * Update test * File should not be a part of this commit * Add centralized shared performance timer for use anywhere * Post-rebase cleanup * Support returning nil from storage provider save * Updates to reflect package changes + other updates in master * Fix storage providers being overwritten * Do not return pointer in save. Support cache headers with S3 providers * Split out videojs + vhs and point to specific working versions of them * Bump vjs and vhs versions * Fix test * Remove unused * Update upload warning message * No longer valid comment * Pin videojs and vhs versions
2020-10-15 00:07:38 +03:00
func GetCacheDurationSecondsForPath(filePath string) int {
if path.Base(filePath) == "thumbnail.jpg" {
// Thumbnails re-generate during live
return 20
} else if path.Ext(filePath) == ".js" || path.Ext(filePath) == ".css" {
// Cache javascript & CSS
return 60
} else if path.Ext(filePath) == ".ts" {
// Cache video segments as long as you want. They can't change.
// This matters most for local hosting of segments for recordings
// and not for live or 3rd party storage.
return 31557600
} else if path.Ext(filePath) == ".m3u8" {
return 0
}
// Default cache length in seconds
return 30
}