Commit graph

54 commits

Author SHA1 Message Date
Gabe Kangas
f4ed7d7453
Create explicit mapping between cpu use levels and codec presets. Closes #2610 2023-01-22 20:45:19 -08:00
Meisam
64d3c37fb2
improve stream testing script (#2549)
* merge testContent.sh into ocContent.sh

* detect ffmpeg

ffmpeg or ffmpeg.exe in path, current dir, or parent dir

* use ocTestStream in api test

* enable verbose logging for api tests

* log ffmpeg version

* change ffmpeg lookup order

* set path properly for using the local ffmpeg

* rm double space from transcoder error logs

* update tests for new video stream

do not test bitrate

* set test stream target to 127.0.0.1

* log ffmpeg path

* update ffmpeg to v4.4.1

* improve logs

* fix ffmpeg installer script

* fix api test runner

* fix logs

* install fonts

* cleanup

* use ocTestStream.sh for all automated tests

* cleanup ocTestStream.sh

* cleanup test/automated/hls/run.sh

* Fix misspell

* fix ffmpeg installer in automated test runners

* spell fix

* cleanup script

* rev quick api tests

* cleanup tmp paths properly in automated tests

* rm unused ffmpeg package

* cleanup

* fix s3 test

* cache ffmpeg bin for automated tests

* shellcheck allow source

* rm missplaced file if backup fails

* use ffmpeg full path

* set lookup path for shellcheck
2023-01-10 18:50:32 -08:00
Gabe Kangas
2675150cdf
Revert "improve stream testing script (#2525)"
This reverts commit e748d8f765.
2023-01-04 14:22:55 -08:00
Meisam
e748d8f765
improve stream testing script (#2525)
* merge testContent.sh into ocContent.sh

* detect ffmpeg

ffmpeg or ffmpeg.exe in path, current dir, or parent dir

* use ocTestStream in api test

* enable verbose logging for api tests

* log ffmpeg version

* change ffmpeg lookup order

* set path properly for using the local ffmpeg

* rm double space from transcoder error logs

* update tests for new video stream

do not test bitrate

* set test stream target to 127.0.0.1

* log ffmpeg path

* update ffmpeg to v4.4.1

* improve logs

* fix ffmpeg installer script

* fix api test runner

* fix logs

* install fonts

* cleanup

* use ocTestStream.sh for all automated tests

* cleanup ocTestStream.sh

* cleanup test/automated/hls/run.sh

* Fix misspell

* fix ffmpeg installer in automated test runners

* spell fix

* cleanup script

* rev quick api tests

* cleanup tmp paths properly in automated tests

* rm unused ffmpeg package

* cleanup
2023-01-04 14:12:05 -08:00
Gabe Kangas
ccaeda88ad
Remove package-comments rule 2022-08-27 20:31:00 -07:00
Gabe Kangas
18a184eeb7
Use bundled images instead of old webroot files 2022-06-20 22:10:55 -07:00
Lerk
f5a5ac006a
Add videotoolbox codec (#1771)
* add videotoolbox codec

* add -realtime flag for medium and below quality

* add quality level to extra arguments

* use variant flags instead of extra arguments

* add videotoolbox test

* fix test
2022-03-15 00:55:06 -07:00
Gabe Kangas
a8167dc9e0
Fix tests after segment filename change 2022-03-10 14:20:44 -08:00
Gabe Kangas
b9a597299f
Simplify segment filenames. Closes #1778
Remove the seconds as an identifier to allow for segments to be written
more than one per second.
2022-03-10 14:14:02 -08:00
Lerk
a7687c861e
Fix "invalid cross-device link" error when running on a different filesystem (#1769)
* fix cross-device rename errors

* fallback to copy instead of always using it

* use tmp dir in data folder

* recreate tmp dir at startup
2022-03-07 16:30:40 -08:00
Gabe Kangas
f272542e40
Persistent HTTP connections seem to work now. Re-enabling. 2022-01-27 20:22:04 -08:00
Gabe Kangas
0d77856cb9
Simplify saving segments to disk 2022-01-27 20:14:19 -08:00
Gabe Kangas
8bc40486f8
Quiet, linter 2022-01-27 14:58:31 -08:00
Gabe Kangas
045a0a2afd
Social features / ActivityPub federation (#1629)
* Support webfinger requests for the live account. Closes https://github.com/owncast/owncast/issues/1193

* Support for actor requests. Returns response for live actor. Closes https://github.com/owncast/owncast/issues/1203

* Handle follow and unfollow requests. Closes
https://github.com/owncast/owncast/issues/1191 and https://github.com/owncast/owncast/issues/1205 and https://github.com/owncast/owncast/issues/1206 and https://github.com/owncast/owncast/issues/1194

* Add basic support for sending out text activities. For https://github.com/owncast/owncast/issues/1192

* Some error handling and passing of dynamic local account names.

* Add hardcoded example image attachment to test post

* Centralize the map of accounts and inboxes

* No longer disable the preview generator based on YP toggle

* Send a federated message to followers when stream starts. For https://github.com/owncast/owncast/issues/1192

* Placeholder for attaching tags

* Add image description

* Save and get to outbox persistence. Return using outbox endpoint for actor

* Pass payloads to be handled through the gochan

* Handle undo follow requests explitly, not all undo requests

* Add API for manually sending simple federated messages. Closes #1215

* Verify inbox requests. Closes #1321

* Add route to fetch a single AP object by ID. For #1329

* Add responses to fediverse nodeinfo requests

* Set and get federation config values for admin

* Handle host-meta requests

* Do not send out message if disabled. Use saved go live message.

* Require AP-compatible content types for AP-related requests

* Rename ap models to apmodels for clarity

* Change how content type matching takes place.

* io -> ioutil

* Add stub delete activity callback

* Handle likes and announces to surface engagement in chat. Part of #1229

* Append url to go live posts

* Do not require specific content types for nodeinfo requests

* Add follow engagement chat message via AP

* add owncast user-agent to requests

* Set note visibility to public (for now)

* Fix saving/fetching a single object

* Add support for x-nodeinfo2 responses

* Point to the dev admin branch for ap

* Bundle in dev admin for testing

* Add error logging

* Add AP middleware back

* Point to the new external compatible logo endpoint

* Clean up more AP logging to help testing

* Tweak go live text and link hashtags

* Fix bug in fetching init time

* Send update actor activities when server details/profile is updated

* Add federation config overview to web client config

* Add additional actor properties

* Make the AP middleware checking more flexible when looking at types

* First pass at remote fediverse follow flow. For #1371

* Added a basic AP actor followers endpoint

* WIP client followers API

* Add profile-page reference to webfinger response

* Add aliases to webfinger response

* Fix content-type returned to be expected activitypub+json

* First pass at followers api

* Point at local dev copy of go-fed/activity

* Add custom toot Hashtag objects to posts

* Store additional user details to followers table

* Fix AP followers endpoint. Closes #1204

* Add owncast hashtag as an invisible tag to go live posts

* Reject AP requests when it is disabled

* Add actor util for generating full account user from person object

* Verify inbox requests before performing any other work

* Accept actor update requests

* Fix linter errors in federation branch

* Migrate AP SQL to sqlc for type safe queries

* Use the @unclearParadigm REST parameter helper

* Fix verifying post ID on AP engagement

* WIP privacy/request approval

* Style the remote follow modal

* First pass at a followers list component w/ mock data. #1370

* Revert "Use the @unclearParadigm REST parameter helper"

This reverts commit c8af8a413f.

* Fix get followers API

* Add support for requiring approval. Closes https://github.com/owncast/owncast/issues/1208

* Handle Applications as Actors partly for PeerTube support

* add temp todo list

* check route on load, this might change later

* style followers

* account for just 1 tab case

* Remove mock data. Allow showing follow button even when there are no external actions defined

* Point to actual followers API

* Support fallback img for follower views

* Remove duplicate verification. Add some additional verbose logging

* Bundle dev admin

* Add type to host-meta webfinger template response

* Tweak remote follow modal content

* WIP federation followers refactor

* Do not send pointer to middleware

* Update admin

* Add setting for toggling displaying fediverse engagement. Closes #1404

* Add in-development admin

* Do not enable cors on admin followers api

* Add db migration for updating messages table

* Enable empty string go live messages to disable

* Remove debug messages

* Rework some ActivityPub handling.

Create new Actor->Person handling.
Create new Actor->Service handling.
Add engagement handlers to send chat events and store event objects.
Store inbound activities to new ap_inbound_activities table.

* Support federated engagement events.

Store them in the messages table and surface them via chat events.

* Support federated event engatement in the chat

* Tweak web UI followers handling

* Point go.mod at remote fork instead of local

* Update admin

* Merged in develop. Couple fixes

* Update dev admin

* Update fedi engagement posts.

- Fix incorrect action text.
- Add action icons.

* Set public as to instead of cc for ap msg

* Updated styling for federated actions in chat

* Add support for blocking federated domains. Closes #1209

* Force checking of https in verify step

* Update dev admin

* Return user scopes in chat history api. Closes #1586

* Update dev admin

* Add AP outbound request worker pool. Closes #1571

* Disable (temporarily?) owncast tag on AP posts

* Consolidate creating activity+notes in outbound AP messages

* Add inbox worker pool. Closes #1570

* Update dev admin bundle

* Clean up some logs

* Re-enable inbound verfication

* Save full IRI to outbox instead of path

* Reject if full IRI is not found in outbox

* Use full ActivityPub user account in chat event

* Fix and expand follower APIs

- Add missing IDs to AP follower endpoints
- Split AP follower endpoints into initial request and pages.
- Support pagination in AP requests.

* Include IRI in error message

* Hide chat toggle when chat is hidden. Closes #1606

* Updates to followers pagination

* Set default go live message

* Remove log

* indirect -> direct import

* Updates for inbound federated event handling.

- Keep track of existing events and reject duplicates.
- Change what is sent to chat for surfing federated engagement.
- Keep track if outbound events are automated "go live" events or not.

* Update chat federated engagement.

* Update dev admin.

* Move from being a person to a bot (service). Closes #1619

* Only set server init date if not already set

* Only save notes to outbox able

* Rework private-mode followers/approvals

* API for returning a list of federated actions for #1573

* Fix too-small follower cells and jumpy tabs. Closes #1616 and closes #1516

* Fix shortcuts getting fired on inputs. Fixes #1489 and #1201

* Add spinner, autoclose + other fixes to follow modal. Fixes #1593

* Fix fetching a single object by IRI

* SendFederationMessage -> SendFederatedMessage

* Autolink and create tag objects from manual posts. Closes #1620

* Update dev admin bundle

* Handle engagement from non-automated/live posts

* Reject federated engagement actions if they do not match a local post

* Update dev admin bundle

* A bunch of cleanup

* Fix unused assignments and logic

* Remove unused function

* Add content warning and sentive content flag if stream is NSFW. Closes #1624

* Disable fetching objects by IRI when in private mode. Closes #1623

* Update the error message of the remote follow dialog. closes #1622

* Update dev admin

* Fix NREs throwing in test content

* Fix query that wasn't properly filtering out hidden messages

* Test against user being disabled instead of message visibility

* Fix automated test NRE

* Update comment

* Adjust federated engagement chat views. Closes #1617

* Add additional index to users table

* Add support for removing followers/requests. Closes #1630

* Reject federated actions from blocked actors. #1631

* Use fallback avatar if it fails to load. Closes #1635

* Fix styling of follower list. Closes #1636

* Add basic blurb stating they should follow the server. Closes #1641

* Update dev admin

* Set default go live message in migration. Closes #1642

* Reset the messages table on 0.0.11 schema migration

* Fix js error with moderation actions. Closes #1621

* Add a bit more clarification on follow modal. Closes #1599

* Remove todos

* Split out actor and domain blocking checks

* Check for errors on default values being set

* Clean up actor rejection due to being blocked

* Update dev admin

* Add colon to error to make it easier to read

* Remove markdown rendering of go live message. Reorganize text. Remove content warning. Closes #1645

* Break out the sort+render messages logic so it can be fired on visibility change. Closes #1643

* Do not send profile updates if federation is disabled

* Save follow references to inbound activities table

* Update dev admin

* Add blocked actor test

* Remove the overloaded term of Follow from social links

* Fix test running in memory only

* Remove "just" in engagement messags

* Replace star with heart for like action.

* Update dev admin

* Explicitly set cc as public

* Remove overly using the stream name in fediverse engagement messages

* Some federated/follow UI tweaks

* Remove explicit cc and bcc as they are not required

* Explicitly set the audience

* Remove extra margin

* Add Join Fediverse button to follow modal. Closes #1651

* Do not allow multiple follows to send multiple events. Closes #1650

* Give events a min height

* Do not allow old posts to be liked/shared. Closes #1652

* Remove value from log message

* Alert followers on private mode toggle

* Ignore clicks to follow button if disabled

* Remove underline from action buttons

* Add moderator icon to join message

* Update admin

* Post-merge remove unused var

* Remove pointing at feature branch

Co-authored-by: Ginger Wong <omqmail@gmail.com>
2022-01-12 13:53:10 -08:00
Gabe Kangas
ba906e2609
Give transcoder a larger codec buffer size.
To give the codec a bit more breathing room with compression to maybe
improve possible blockiness in high movement scenes.
2021-11-30 13:40:09 -08:00
Gabe Kangas
b052fa410b Update tests 2021-11-26 21:05:18 -08:00
Gabe Kangas
73e58a7801 Refactor the offline clip handling.
More stable, reduced function complexity.
2021-11-26 20:55:31 -08:00
Gabe Kangas
3585340f1b Update test 2021-11-23 22:02:39 -08:00
Eng Zer Jun
7996df85e1
refactor: move from io/ioutil to io and os packages (#1546)
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2021-11-19 22:42:50 -08:00
Gabe Kangas
4c37308e71 Update the HLS flags to adhere to more Apple recommendations 2021-11-01 00:20:08 -07:00
Gabe Kangas
5b38426e31 Tweak video bandwidth values to be < 10% over max 2021-10-31 23:00:11 -07:00
Gabe Kangas
bdce2e13bf Some linter cleanup 2021-10-25 00:31:45 -07:00
Gabe Kangas
3f4176d8d6 Replace panics with our logging framework 2021-10-25 00:15:58 -07:00
Gabe Kangas
c4d2ffd834
Fix crash when an invalid inbound stream is disconnected. Closes #1439 (#1483) 2021-10-21 17:21:17 -07:00
Gabe Kangas
ca9d5de192
Embed static resources (#1466)
* Replace pkger with go:embed for bundling the admin. Closes #844

* Remove references to pkged.go

* Point tests to use an updated version of Go

* Add comment to new exported function

* Cleanup

* Add a dummy pkged.go to alert people to stop using it.

* Add simple browser test to make sure the admin is available and renders

* Don't panic

* Embed bot/scraper metadata template.

Add browser test to validate the rendering of this template.

* Use embedded offline.ts segment

* Remove placeholder thumbnail as its unnecessary

* Remove copying the static directory into the release

* Cleanup
2021-10-11 15:04:16 -07:00
Gabe Kangas
1b053ffd1b
Simplify HLS storage paths (#1393)
* Remove private vs public HLS paths and add a HLS controller. Closes #875

* Use http.ServeFile instead
2021-09-12 11:32:42 -07:00
Gabe Kangas
c6c6f0233d
Expanded linting + fix warnings (#1396)
* Expand the linters and types of warnings to improve consistency and safety

* Fail lint workflow if there are errors

* golint has been replaced by revive

* Hand-pick some of the default exclude list

* Ignore error when trying to delete preview gif

* Ignore linter warning opening playlist path

* Rename user field Id -> ID

* A bunch of renames to address linter warnings

* Rename ChatClient -> Client per linter suggestion best practice

* Rename ChatServer -> Server per linter suggestion best practice

* More linter warning fixes

* Add missing comments to all exported functions and properties
2021-09-12 00:18:15 -07:00
Halil İbrahim Kocaöz
d90b02e50b
fix moved transcoding error links (#1381) 2021-09-05 18:07:02 -07:00
ooa113y
efb80243d5
Fix Windows tests (#1376)
* Fix Windows tests

* Use filepath.Join
2021-09-04 21:17:15 -07:00
Meisam
41a7e8b896
check the return value of os.Rename (#1288) 2021-07-31 14:59:02 -07:00
Christian
cb7a9b89ea
safely generate the thumbnail.jpg & preview.gif (#1279) 2021-07-28 14:21:02 -07:00
Gabe Kangas
3dc3ad75f6 Cleanup errors that are not fatal during filesystem cleanup 2021-07-22 16:50:30 -07:00
Gabe Kangas
44d6a36b77 Cleanup linter warnings 2021-07-19 23:49:16 -07:00
Gabe Kangas
37fedb8ab8 Ignore some additional non-fatal errors 2021-07-18 13:10:08 -07:00
Meisam
7361578412
style fix for error checking (#1170)
+additional linting
2021-07-09 11:16:44 -07:00
Meisam
a13e1e75e2
proper cleanup and permission check for the hls directory (#1167)
* check error on hls cleanup

* rm HLS directories before creating new ones

* don't mask the variable

* mv cleanupDirectory() to utils

* add user-friendly error messages
2021-07-08 12:35:53 -07:00
Jannik
3f9f4a151c
refactor: use io.Pipe and ffmpeg's stdin (#1148)
This removes the usage of `syscall.Mkfifo` which was previously used and
won't work on Windows systems and opens the door for other processes on
the computer to interfere in the rtmp stream (dumping bad content in the
fifo, removing the file, blocking the file in offline status).
Instead, this patch introduces an `io.Pipe` which pipes the RTMP stream
to the ffmpeg command while staying in Owncast.

Further links:
* ffmpeg on using `pipe:0` as an input: https://ffmpeg.org/ffmpeg-protocols.html#pipe
2021-07-03 12:28:25 -07:00
Gabe Kangas
12104978e8 Some linter cleanup 2021-06-29 10:21:00 -07:00
Gabe Kangas
1a0beb1d0e Fix tests 2021-05-23 14:27:35 -07:00
Gabe Kangas
381daaec9b Save owncast logs to log file. Closes #908 2021-05-22 19:25:54 -07:00
tomleb
1504ea3509
Add -logdir flag (#1039)
This allow configuring the various logs that may be outputted
(transcoder and future logs)
2021-05-22 16:29:49 -07:00
Gabe Kangas
09da85090e Fix tests 2021-05-05 19:58:02 -07:00
Gabe Kangas
4f4cef97e2 Force an iframe every segment 2021-05-05 18:21:27 -07:00
Gabe Kangas
fc123ecda3 Fix tests 2021-05-05 17:31:19 -07:00
Gabe Kangas
fa3a959dc5 update tests 2021-04-29 22:13:47 -07:00
Gabe Kangas
5324d235bf Force an i-frame every second for consistent segment lenghts with
minimum latency.
2021-04-29 22:06:39 -07:00
Gabe Kangas
eedc07fc94 Ignore warning about hls segment length 2021-04-24 14:00:35 -07:00
Gabe Kangas
34c4350597 Add some more va-api codec error mappings 2021-04-21 17:48:39 -07:00
Gabe Kangas
a06a9c58ea Add i965-va-driver-shader missing error 2021-04-15 22:25:28 -07:00
Gabe Kangas
543577c2e0 Create hls directories at transcoder start to account for stream output changes. Fixes #940 2021-04-15 21:36:14 -07:00