Commit graph

174 commits

Author SHA1 Message Date
kim
b84637801a
[chore] update go ffmpreg to v0.6.0 ()
* pull in go-ffmpreg v0.6.0

* add code comment

* grrr linter

* set empty module name when calling ffmpeg / ffprobe
2024-11-06 14:38:13 +01:00
kim
53aaeb18d4
previously we were using the ffmpeg runner for ffprobe 🤦 () 2024-11-05 10:50:56 +00:00
kim
8f288f1689
[bugfix] determine mime-type to use during ffprobe evaluation stage, don't bother checking against file extension ()
* determine mime-type to use during ffprobe evaluation stage, don't bother rechecking by file extension

* set mjpeg content-type

* fix up tests expecting differing default values
2024-11-04 14:58:15 +01:00
kim
7ec6509e11
[bugfix] support classifying correct video codec without audio as webm ()
* for webm support video:[vp8,vp9,av1] and audio:[NONE,vorbis,opus]

* improved unsupported data type error output
2024-10-28 14:09:21 +01:00
kim
e86592bc32
[chore] pull in go-ffmpreg v0.4.1 ()
* pull in go-ffmpreg v0.4.1

* bring back GTS_WAZERO_COMPILATION_CACHE
2024-10-28 10:55:48 +00:00
tobi
602c858379
[chore] Thumbnail only first frame of animated media () 2024-10-18 15:44:08 +02:00
Markus Unterwaditzer
a48cce82b9
[chore] Upgrade golangci-lint, ignore existing int overflow warnings ()
* [chore] Bump tooling versions, bump go -> v1.23.0

* undo silly change

* sign

* bump go version in go.mod

* allow overflow in imaging

* goreleaser deprecation notices

* [chore] Upgrade golangci-lint, ignore existing int overflow warnings

There is a new lint for unchecked int casts. Integer overflows are bad,
but the old code that triggers this lint seems to be perfectly fine.
Instead of disabling the lint entirely for new code as well, grandfather
in existing code.

* fix golangci-lint documentation link

* revert unrelated changes

* revert another unrelated change

* get rid of remaining nolint:gosec

* swagger updates

* apply review feedback

* fix wrong formatting specifier thing

* fix the linter for real

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-10-16 14:13:58 +02:00
kim
6a76b9d609
[feature/OFFICIALLY UNSUPPORTED] add nowasm build tag to disable building with WebAssembly ()
* add experimental build-tag 'nowasm' which uses local ffmpeg / ffprobe

* updated experimental support message

* add comment to build script explaining build tag

* add nowasm build tags to moderncsqlite files
2024-10-14 11:59:12 +02:00
kim
bd1866ad8a
update go-ffmpreg to v0.3.1 (pulls in latest wazero too) () 2024-10-06 20:53:03 +00:00
kim
6f3583559f
[bugfix] better handle ogg container format () 2024-09-27 13:16:34 +02:00
kim
497ebd8c4e
[bugfix] fix media limit reader check ()
* return nicer errors for frontend when media / emoji upload limit reached

* fix reader limit check

* add code comment
2024-09-27 13:15:53 +02:00
kim
53ee6aef08
[bugfix] s3 media uploaded without content-type ()
* update go-storage dependency, for S3Storage manually call PutObject() so we can set content-type

* update calls to PutFile() to include the contentType
2024-09-26 14:43:10 +02:00
kim
dc4059e9a2
[bugfix] add support for media with rotation contained in stream side data ()
* add support for media with embedded rotation data in stream side data list

* *grumble grumble* linter
2024-09-23 15:13:01 +00:00
kim
0a1555521d
[performance] use single-threaded image transforms ()
* use single-threaded image resizing in native code so we have more control over goroutines

* implement parallel-free versions of image transform functions also

* remove debug code
2024-08-31 10:41:38 +02:00
tobi
cd93a5baf3
[security] Implement allowFiles fs for better isolation of ffmpeg / ffprobe ()
* [chore] Implement readOneFile fs

* further isolation

* remove fmt call

* tweaks
2024-08-30 14:03:59 +02:00
tobi
e10aa76612
[chore] Only call imaging.Resize when necessary, use even tinier blurhashes ()
* [chore] Use `imaging.Fit`, use even tinier blurhashes

* avoid calling resize if not necessary

* update blurhashes + thumb
2024-08-29 17:43:14 +02:00
tobi
db59ca36b1
[chore] Remove unused "env" module () 2024-08-25 15:37:20 +00:00
kim
8e5a72ac5c
[performance] ffmpeg ffprobe wrapper improvements ()
* use a single instance of wazero runtime and compiled modules

* remove test output 🤦

* undo process-{media,emoji} changes

* update test runner to include wazero compilation cache

* sign drone.yml

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-23 17:15:35 +02:00
kim
889d4756ea
[performance] use native Go code to probe JPEGs ()
* use native Go code to probe JPEGs

* add note about copying from github.com/disintegration/imaging

* add more code comments
2024-08-19 13:38:10 +00:00
kim
ae14aa004d
[bugfix] permit unspecified orientation data () 2024-08-16 14:10:03 +02:00
kim
f77005128a
[performance] move thumbnail generation to go code where possible ()
* wrap thumbnailing code to handle generation natively where possible

* more code comments!

* add even more code comments!

* add code comments about blurhash generation

* maintain image rotation if contained in exif data

* move rotation before resizing

* ensure pix_fmt actually selected by ffprobe, check for alpha layer with gifs

* use linear instead of nearest-neighbour for resizing

* work with image "orientation" instead of "rotation". use default 75% quality for both webp and jpeg generation

* add header to new file

* use thumb extension when getting thumb mime type

* update test models and tests with new media processing

* add suggested code comments

* add note about thumbnail filter count reducing memory usage
2024-08-08 17:12:13 +00:00
tobi
b19cfee7ae
[feature] Use gifv type for short soundless mp4 videos () 2024-08-08 08:12:16 +00:00
kim
b85a9983d0
[bugfix] fix emoji recaching operations ()
* add test for emoji update image

* update emoji recache to set the instance account id

* don't refresh emoji if only not cached. in that case literally just recache

* code comment

* rename + move a few things

* add some more code comments, and rename some functions to make logic a bit clearer

* remove unnecessary nil check (the value can be nil)

* comment wording

* remove test data output

* handle the case of caching an emoji which has been refreshed then uncached

* allow overwriting on testrig storage as we do now on regular storage

* fix emoji category ID not getting updated

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-03 19:05:38 +02:00
kim
e5e996b28a
[bugfix] close files before error return ()
* close files before error return

* use defer statements

* shuffle around some defers
2024-08-02 15:11:24 +01:00
kim
94e87610c4
[chore] add back exif-terminator and use only for jpeg,png,webp ()
* add back exif-terminator and use only for jpeg,png,webp

* fix arguments passed to terminateExif()

* pull in latest exif-terminator

* fix test

* update processed img

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-08-02 12:46:41 +01:00
tobi
697261da53
[chore] Take account of rotation data when calculating full size image dimensions ()
* [chore] Take account of rotation data when calculating full size image dimensions

* boobies
2024-07-31 18:43:39 +00:00
kim
0b4a951b48
[performance] limit ffprobe read time to at most 1s after start of file () 2024-07-30 16:18:29 +02:00
kim
47c26818d6
[chore] add some more slice related utility functions + remove duplicated functions () 2024-07-30 11:29:32 +02:00
kim
368c97f0f8
[bugfix] take into account rotation when generating thumbnail ()
* take into account rotation when generating thumbnail, simplify ffprobe output to show only fields we need

* only show rotation side data

* remove unnecessary comment

* fix code comments

* remove debug logging
2024-07-28 21:10:41 +02:00
kim
58f8082795
[bugfix] moves file rename to earlier in media pipeline so ffmpeg calls ALWAYS have extension () 2024-07-28 10:31:49 +02:00
kim
d9e70b942f
ensure ffmpeg muxer gets set correctly by setting input file extension before clean () 2024-07-26 16:30:24 +01:00
kim
31294f7c78
[bugfix] media.Processor{}.GetFile() returning 404s on first call, correctly loading on 2nd ()
* refactor file handling a tiny bit

* whoops

* make processing media / emoji defers a bit clear to see that it's the "on finished processing" path

* some wording

* add some debug logging

* add mutex locks for processing remote media

* try removing freshness check

* fix derefMedia not being allocated

* fix log format string

* handle case of empty file paths (i.e. not stored)

* remove media / emoji once finished processing from dereferencer maps

* whoops, fix the cached / force checks

* move url parsing outside of 'process___Safely()' funcs to prevalidate url

* use emoji.ShortcodeDomain()

* update RefreshEmoji() to also match RefreshMedia() changes

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-22 18:45:48 +01:00
kim
c635080a0b
[feature] add flac support ()
* add flac support to the ffprobe format/stream -> filetype parser

* also add audio/flac for flac (not just x-flac)

* update tests

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-21 14:42:51 +02:00
kim
409b3987ac
update go-ffmpreg to v0.2.3 () 2024-07-20 14:14:58 +01:00
kim
50c9b5498b
[feature] use webp for thumbnails ()
* update to use webp for thumbnails

* bump webp quality up to 40% from 12% (it's a bit different to jpeg quality setting)

* update to use yuva colorspace, and use thumbnail=n=10 to select frame

* fix missing comma in ffmpeg args

* add links to appropriate ffmpeg docs

* update tests

* add file size tests for thumbnails

---------

Co-authored-by: tobi <tobi.smethurst@protonmail.com>
2024-07-19 17:28:43 +02:00
kim
72ba5666a6
[chore] media pipeline improvements ()
* don't set emoji / media image paths on failed download, migrate FileType from string to integer

* fix incorrect uses of util.PtrOr, fix returned frontend media

* fix migration not setting arguments correctly in where clause

* fix not providing default with not null column

* whoops

* ensure a default gets set for media attachment file type

* remove the exclusive flag from writing files in disk storage

* rename PtrOr -> PtrOrZero, and rename PtrValueOr -> PtrOrValue to match

* slight wording changes

* use singular / plural word forms (no parentheses), is better for screen readers

* update testmodels with unknown media type to have unset file details, update attachment focus handling converting to frontend, update tests

* store first instance in ffmpeg wasm pool, fill remaining with closed instances
2024-07-17 15:26:33 +00:00
kim
401098191b
give read-only access to /dev for ffmpeg to access /dev/urandom () 2024-07-16 13:32:48 +01:00
kim
de45c0be60
[feature] more filetype support! ()
* add more supported file types to our media processor that ffmpeg supports, update supported mime type lists

* add code comments to the supported mime types slice

* don't check for zero value string, just parse

* remove some unneeded consts which make the code a bit harder to read

* fix test expected instance media mime types, use compact ffprobe json, simple media processing by type

* final tweaks to media processing code

* don't use safe divide where we don't need to
2024-07-15 15:24:53 +01:00
tobi
9efb11d848
[feature/frontend] Add player for audio files; use thumbnail for poster ()
* [feature/frontend] Audio player for audio media types

* use video preview images for previews instead of video itself

* don't preload

* update tests for new zork status

* collapse media gallery into single row when small
2024-07-15 10:47:57 +01:00
kim
cde2fb6244
[feature] support processing of (many) more media types ()
* initial work replacing our media decoding / encoding pipeline with ffprobe + ffmpeg

* specify the video codec to use when generating static image from emoji

* update go-storage library (fixes incompatibility after updating go-iotools)

* maintain image aspect ratio when generating a thumbnail for it

* update readme to show go-ffmpreg

* fix a bunch of media tests, move filesize checking to callers of media manager for more flexibility

* remove extra debug from error message

* fix up incorrect function signatures

* update PutFile to just use regular file copy, as changes are file is on separate partition

* fix remaining tests, remove some unneeded tests now we're working with ffmpeg/ffprobe

* update more tests, add more code comments

* add utilities to generate processed emoji / media outputs

* fix remaining tests

* add test for opus media file, add license header to utility cmds

* limit the number of concurrently available ffmpeg / ffprobe instances

* reduce number of instances

* further reduce number of instances

* fix envparsing test with configuration variables

* update docs and configuration with new media-{local,remote}-max-size variables
2024-07-12 09:39:47 +00:00
Vyr Cossont
fde0c6bc8c
[chore] Allow gtsmodel to depend on util ()
Convert one free function into a gtsmodel.Emoji method so that util does not depend on gtsmodel and can be used from it in the future
2024-07-03 15:53:54 -07:00
kim
21bb324156
[chore] media and emoji refactoring ()
* start updating media manager interface ready for storing attachments / emoji right away

* store emoji and media as uncached immediately, then (re-)cache on Processing{}.Load()

* remove now unused media workers

* fix tests and issues

* fix another test!

* fix emoji activitypub uri setting behaviour, fix remainder of test compilation issues

* fix more tests

* fix (most of) remaining tests, add debouncing to repeatedly failing media / emojis

* whoops, rebase issue

* remove kim's whacky experiments

* do some reshuffling, ensure emoji uri gets set

* ensure marked as not cached on cleanup

* tweaks to media / emoji processing to handle context canceled better

* ensure newly fetched emojis actually get set in returned slice

* use different varnames to be a bit more obvious

* move emoji refresh rate limiting to dereferencer

* add exported dereferencer functions for remote media, use these for recaching in processor

* add check for nil attachment in updateAttachment()

* remove unused emoji and media fields + columns

* see previous commit

* fix old migrations expecting image_updated_at to exists (from copies of old models)

* remove freshness checking code (seems to be broken...)

* fix error arg causing nil ptr exception

* finish documentating functions with comments, slight tweaks to media / emoji deref error logic

* remove some extra unneeded boolean checking

* finish writing documentation (code comments) for exported media manager methods

* undo changes to migration snapshot gtsmodels, updated failing migration to have its own snapshot

* move doesColumnExist() to util.go in migrations package
2024-06-26 16:01:16 +01:00
kim
3d3e99ae52
[performance] update storage backend and make use of seek syscall when available ()
* update to use go-storage/ instead of go-store/v2/storage/

* pull in latest version from codeberg

* remove test output 😇

* add code comments

* set the exclusive bit when creating new files in disk config

* bump to actual release version

* bump to v0.1.1 (tis a simple no-logic change)

* update readme

* only use a temporary read seeker when decoding video if required (should only be S3 now)

* use fastcopy library to use memory pooled buffers when calling TempFileSeeker()

* update to use seek call in serveFileRange()
2024-05-22 11:46:24 +02:00
kim
c9c0773f2c
[performance] update remaining worker pools to use queues ()
* start replacing client + federator + media workers with new worker + queue types

* refactor federatingDB.Delete(), drop queued messages when deleting account / status

* move all queue purging to the processor workers

* undo toolchain updates

* code comments, ensure dereferencer worker pool gets started

* update gruf libraries in readme

* start the job scheduler separately to the worker pools

* reshuffle ordering or server.go + remove duplicate worker start / stop

* update go-list version

* fix vendoring

* move queue invalidation to before wipeing / deletion, to ensure queued work not dropped

* add logging to worker processing functions in testrig, don't start workers in unexpected places

* update go-structr to add (+then rely on) QueueCtx{} type

* ensure more worker pools get started properly in tests

* fix remaining broken tests relying on worker queue logic

* fix account test suite queue popping logic, ensure noop workers do not pull from queue

* move back accidentally shuffled account deletion order

* ensure error (non nil!!) gets passed in refactored federatingDB{}.Delete()

* silently drop deletes from accounts not permitted to

* don't warn log on forwarded deletes

* make if else clauses easier to parse

* use getFederatorMsg()

* improved code comment

* improved code comment re: requesting account delete checks

* remove boolean result from worker start / stop since false = already running or already stopped

* remove optional passed-in http.client

* remove worker starting from the admin CLI commands (we don't need to handle side-effects)

* update prune cli to start scheduler but not all of the workers

* fix rebase issues

* remove redundant return statements

* i'm sorry sir linter
2024-04-26 13:50:46 +01:00
kim
a483bd9e38
[performance] massively improved ActivityPub delivery worker efficiency ()
* add delivery worker type that pulls from queue to httpclient package

* finish up some code commenting, bodge a vendored activity library change, integrate the deliverypool changes into transportcontroller

* hook up queue deletion logic

* support deleting queued http requests by target ID

* don't index APRequest by hostname in the queue

* use gorun

* use the original context's values when wrapping msg type as delivery{}

* actually log in the AP delivery worker ...

* add uncommitted changes

* use errors.AsV2()

* use errorsv2.AsV2()

* finish adding some code comments, add bad host handling to delivery workers

* slightly tweak deliveryworkerpool API, use advanced sender multiplier

* remove PopCtx() method, let others instead rely on Wait()

* shuffle things around to move delivery stuff into transport/ subpkg

* remove dead code

* formatting

* validate request before queueing for delivery

* finish adding code comments, fix up backoff code

* finish adding more code comments

* clamp minimum no. senders to 1

* add start/stop logging to delivery worker, some slight changes

* remove double logging

* use worker ptrs

* expose the embedded log fields in httpclient.Request{}

* ensure request context values are preserved when updating ctx

* add delivery worker tests

* fix linter issues

* ensure delivery worker gets inited in testrig

* fix tests to delivering messages to check worker delivery queue

* update error type to use ptr instead of value receiver

* fix test calling Workers{}.Start() instead of testrig.StartWorkers()

* update docs for advanced-sender-multiplier

* update to the latest activity library version

* add comment about not using httptest.Server{}
2024-04-11 11:45:35 +02:00
dependabot[bot]
7bc536d1f7
[chore]: Bump github.com/stretchr/testify from 1.8.4 to 1.9.0 ()
* [chore]: Bump github.com/stretchr/testify from 1.8.4 to 1.9.0

Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.4 to 1.9.0.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.4...v1.9.0)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

* cast the exxpected test values as float32 to bypass (possible) testify suite regression

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kim <grufwub@gmail.com>
2024-03-04 15:05:24 +00:00
tobi
9cadc764b3
[feature] Add experimental instance-federation-spam-filter option ()
* [chore] Move `visibility` to `filter/visibility`

* [feature] Add experimental instance-federation-spam-filter option
2024-02-27 12:22:05 +00:00
tobi
aa8bbe6ad2
[bugfix] Fix Postgres emoji delete, emoji category change ()
* [bugfix] Fix Postgres emoji delete, emoji category change

* revert trace logging

* caching issue

* update tests
2024-01-29 14:57:22 +00:00
kim
7ec1e1332e
[performance] overhaul struct (+ result) caching library for simplicity, performance and multiple-result lookups ()
* rewrite cache library as codeberg.org/gruf/go-structr, implement in gotosocial

* use actual go-structr release version (not just commit hash)

* revert go toolchain changes (damn you go for auto changing this)

* fix go mod woes

* ensure %w is used in calls to errs.Appendf()

* fix error checking

* fix possible panic

* remove unnecessary start/stop functions, move to main Cache{} struct, add note regarding which caches require start/stop

* fix copy-paste artifact... 😇

* fix all comment copy-paste artifacts

* remove dropID() function, now we can just use slices.DeleteFunc()

* use util.Deduplicate() instead of collate(), move collate to util

* move orderByIDs() to util package and "generify"

* add a util.DeleteIf() function, use this to delete entries on failed population

* use slices.DeleteFunc() instead of util.DeleteIf() (i had the logic mixed up in my head somehow lol)

* add note about how collate differs from deduplicate
2024-01-19 12:57:29 +00:00
tobi
ebf550b7c1
[chore] Move to codeberg's exif-terminator () 2024-01-15 20:54:32 +00:00