Commit graph

1686 commits

Author SHA1 Message Date
Daenney
0cce2c0838
[feature] Block a bunch of "AI" crawlers (#2239)
* [feature] Block Google Bard/AI crawlers

* [feature] Block the other OpenAI crawler

* [feature] Block Common Crawl crawler

This is used in research, but also gleefully advertises itself as the
training source used in all LLMs and GPT-3.

Fixes: #2240

* [feature] Block Omgilikebot

Used by some shady big web data engine company.

* [feature] Block Meta's language model crawler

* [feature] Block well-known.dev crawler
2023-09-30 20:44:57 +01:00
tobi
2b6b9cdf83
[bugfix] Fix paging for empty items (#2236)
* use minID properly for public timeline

* return paged response properly even when 0 items

* use gtserror

* page more consistently (for now)

* test

* aaa
2023-09-29 15:31:10 +02:00
tobi
736cd37caf
[frontend] Add discoverable flag to settings panel (#2235) 2023-09-29 13:01:36 +02:00
tobi
536d9e482d
[chore/bugfix] Deinterface text.Formatter, allow underscores in hashtags (#2233) 2023-09-29 10:39:56 +02:00
tobi
b6b8f82c87
[bugfix] Move follow.show_reblogs check further up to avoid showing unwanted reblogs in home timeline (#2234) 2023-09-29 10:39:35 +02:00
Daenney
2a9927dfdc
[chore] Enable S256 code challenge (#2224)
For some reason we only had plain enabled. This adds S256.
2023-09-28 11:21:19 +02:00
dependabot[bot]
e93cc7a9a8
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc (#2221)
Bumps [go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc](https://github.com/open-telemetry/opentelemetry-go) from 1.17.0 to 1.18.0.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.17.0...v1.18.0)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-28 11:20:26 +02:00
dependabot[bot]
d45a75e475
[chore]: Bump golang.org/x/oauth2 from 0.11.0 to 0.12.0 (#2223)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-25 11:52:59 +01:00
dependabot[bot]
380d83f9a9
[chore]: Bump github.com/abema/go-mp4 from 0.13.0 to 1.0.0 (#2222)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-25 11:33:10 +01:00
kim
9f9fcf743d
[bugfix] support both CollectionPage AND OrderedCollectionPage in status replies (#2220) 2023-09-23 18:28:12 +01:00
kim
8f67dd583d
[chore] deinterface the typeutils.Converter and update to use state structure (#2217)
* update typeconverter to use state structure

* deinterface the typeutils.TypeConverter -> typeutils.Converter

* finish copying over old type converter code comments

* fix cherry-pick merge issues, fix tests pointing to old typeutils interface type still
2023-09-23 18:44:11 +02:00
K
1b848aa1e5
[chore] fix typo in slice.go (#2219) 2023-09-23 18:42:34 +02:00
tobi
b987653cc9
[docs] ahs article milestone a (#2218) 2023-09-22 11:25:25 +02:00
tobi
183eaa5b29
[feature] Implement explicit domain allows + allowlist federation mode (#2200)
* love like winter! wohoah, wohoah

* domain allow side effects

* tests! logging! unallow!

* document federation modes

* linty linterson

* test

* further adventures in documentation

* finish up domain block documentation (i think)

* change wording a wee little bit

* docs, example

* consolidate shared domainPermission code

* call mode once

* fetch federation mode within domain blocked func

* read domain perm import in streaming manner

* don't use pointer to slice for domain perms

* don't bother copying blocks + allows before deleting

* admonish!

* change wording just a scooch

* update docs
2023-09-21 12:12:04 +02:00
Daenney
d6add4ef93
[docs] Add a note on cluster support (#2214)
* [docs] Callout no cluster/multi-node support

    Though we have a GitHub issue for this, we don't mention it anywhere in
    the docs. This adds it to the deployment considerations so hopefully
    folks will see it when standing up their own instance.

* [docs] Add daenney as a developer
2023-09-20 18:12:31 +02:00
kim
fc11deeb83
[feature] add paging to AP following / followers endpoints (#2198) 2023-09-20 16:49:46 +01:00
dependabot[bot]
c6fdcd52fa
[chore]: Bump go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp from 1.17.0 to 1.18.0 (#2207)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 13:47:28 +01:00
dependabot[bot]
f302ebb8e5
[chore]: Bump github.com/miekg/dns from 1.1.55 to 1.1.56 (#2204)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-18 13:45:20 +01:00
kim
ba0a31d224
[bugfix] fix flakey paging test (#2210) 2023-09-18 13:45:10 +01:00
kim
50b713e37a
[chore] bump bun (and related libraries) versions to v1.1.16 (#2209) 2023-09-18 11:38:56 +01:00
kim
7293d6029b
[feature] add paging to account follows, followers and follow requests endpoints (#2186) 2023-09-12 14:00:35 +01:00
tobi
4b594516ec
[feature] Allow admins to expire remote public keys; refetch expired keys on demand (#2183) 2023-09-12 10:43:12 +01:00
Daenney
2cac5a4613
[feature] Support Actor URIs for webfinger queries (#2187)
* [feature] Support Actor URIs for webfinger queries

It's now possible to pass an Actor URI as the resource to query for when
doing a webfinger query. The code now extracts the username and domain
from the URI. The URI needs to be fully qualified, including having a
scheme of http or https to be recognised as such.

The acct scheme is handled as we used to, including dealing with an
erroneous leading @ on the username. We retain the ability to handle
resources without a scheme by parsing them again with the acct scheme if
the original parse failed. This can happen due to parsing ambiguities
when dealing with a string like user@domain.tld:port.

* [bugfix] Remove debugging changes

* [chore] Make TestExtractNamestring table-driven

* [chore] Unnest Trim and Split for readability
2023-09-11 18:38:31 +02:00
tobi
7011f57b09
[chore] bump bun v1.1.14 -> v1.1.15 (#2195) 2023-09-11 15:16:52 +02:00
dependabot[bot]
23dd6f664c
[chore]: Bump golang.org/x/net from 0.14.0 to 0.15.0 (#2193)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.14.0 to 0.15.0.
- [Commits](https://github.com/golang/net/compare/v0.14.0...v0.15.0)

---
updated-dependencies:
- dependency-name: golang.org/x/net
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-11 12:47:44 +02:00
dependabot[bot]
ecbce17655
[chore]: Bump github.com/tdewolff/minify/v2 from 2.12.8 to 2.12.9 (#2194) 2023-09-11 09:50:58 +00:00
dependabot[bot]
f0a3fcae85
[chore]: Bump golang.org/x/crypto from 0.12.0 to 0.13.0 (#2190) 2023-09-11 09:50:37 +00:00
OniriCorpe
79906f969d
[docs] add fail2ban regex in the doc (#2189) 2023-09-11 09:43:31 +02:00
kim
b093947d84
[chore] much improved paging package (#2182) 2023-09-07 15:58:37 +01:00
Daenney
14ef098099
[feature] Support OTLP HTTP, drop Jaeger (#2184)
* [feature] Add http trace exporter, drop Jaeger

Jaeger supports ingesting traces using the OpenTelemetry gRPC or HTTP
methods. The Jaeger project has deprecated the old jaeger transport.

* Add support for submitting traces over HTTP
* Drop support for the old Jaeger protocol
* Upgrade the trace libraries to v1.17

Fixes: #2176
Fixes: #2179
2023-09-07 13:20:37 +02:00
kim
916c6d07ba
[bugfix] fix checks for deref the same status descendants / ascendants (#2181) 2023-09-05 11:22:02 +01:00
tobi
9f2199f9a9
[chore] Add script for downloading latest snapshot tar.gz (#2157)
* [chore] Add script for downloading latest snapshot tar.gz

* Update scripts/get_latest_snapshot.sh

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* Update scripts/get_latest_snapshot.sh

Co-authored-by: Daenney <daenney@users.noreply.github.com>

* update comment

---------

Co-authored-by: Daenney <daenney@users.noreply.github.com>
2023-09-04 19:18:39 +02:00
Daenney
5f10885dde
[feature] Don't trace non-existing routes (#2172)
c.FullPath() is the empty string if a request doesn't match any route on
our mux. In those cases, there's no value in emitting a trace. The trace
will be empty, containing no other information beyond the fact that we
didn't match a route. Since Gin breaks off the processing early we don't
need to trace this request as it won't do anything and consumes no
further resources.

The 404 will still be emitted by our logs and will be visible from a
reverse proxy too.
2023-09-04 17:15:14 +02:00
tobi
3ed1ca68e5
[feature] Store admin actions in the db, prevent conflicting actions (#2167) 2023-09-04 14:55:17 +01:00
dependabot[bot]
4f3daeebcb
[chore]: Bump golang.org/x/image from 0.11.0 to 0.12.0 (#2178)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 12:51:12 +01:00
dependabot[bot]
c0bddd272f
[chore]: Bump github.com/minio/minio-go/v7 from 7.0.62 to 7.0.63 (#2180)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:16:41 +01:00
dependabot[bot]
ddd3c2e44b
[chore]: Bump golang.org/x/text from 0.12.0 to 0.13.0 (#2177)
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-09-04 10:16:06 +01:00
kim
4eb77ff5d7
[bugfix] move SQLite pragmas into connection string (#2171)
* move SQLite pragmas into connection string

Signed-off-by: kim <grufwub@gmail.com>

* use url.Values type for SQLite connection preferences

Signed-off-by: kim <grufwub@gmail.com>

* set SQLite URI prefs properly using _pragma query key

Signed-off-by: kim <grufwub@gmail.com>

* add notes on SQLite connection preferences

Signed-off-by: kim <grufwub@gmail.com>

* fix typo

Signed-off-by: kim <grufwub@gmail.com>

* add one extra line regarding connection pooling

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-09-01 15:13:33 +02:00
kim
1ee99fc165
[bugfix] wrap bun.Tx to add our own error processing (#2169)
* wrap bun.Tx to add our own error processing

Signed-off-by: kim <grufwub@gmail.com>

* add compile-time check for updateRowError() compatibility with sql.Row, fix wrapTx() not being used properly

Signed-off-by: kim <grufwub@gmail.com>

---------

Signed-off-by: kim <grufwub@gmail.com>
2023-08-31 11:46:15 +02:00
dependabot[bot]
2ec313a21f
[chore]: Bump github.com/google/uuid from 1.3.0 to 1.3.1 (#2166) 2023-08-28 07:12:40 +00:00
dependabot[bot]
e6407ec95c
[chore]: Bump github.com/KimMachineGun/automemlimit from 0.2.6 to 0.3.0 (#2165) 2023-08-28 06:59:08 +00:00
tobi
13f1c85e70
[chore/frontend] Make line-height a wee little bit bigger (#2159) 2023-08-24 12:52:12 +02:00
Daenney
5416ad9888
[docs] Update backup docs (#2153) 2023-08-23 19:06:46 +01:00
tobi
083e8f35b3
[bugfix/frontend] Normalize header sizes (#2152)
* [bugfix/frontend] Normalize header sizes

* ensmallen heading a little
2023-08-23 18:28:59 +02:00
rdelaage
7b48437f17
[feature] list commands for both attachment and emojis (#2121)
* [feature] list commands for both attachment and emojis

* use fewer commands, provide `local-only` and `remote-only` as filters

* envparsing

---------

Co-authored-by: Romain de Laage <romain.delaage@rdelaage.ovh>
Co-authored-by: tsmethurst <tobi.smethurst@protonmail.com>
2023-08-23 18:01:16 +02:00
tobi
8f38dc2e7f
[feature] Add rate limit exceptions option, use ISO8601 for rate limit reset (#2151)
* start updating rate limiting, add exceptions

* tests, comments, tidying up

* add rate limiting exceptions to example config

* envparsing

* nolint

* apply kimbediff

* add examples
2023-08-23 14:32:27 +02:00
tobi
94d16631bc
[performance] Rework home timeline query to use cache more (#2148) 2023-08-22 15:41:51 +02:00
Daenney
4ae16bce8c
[feature] Make log format configurable (#2130)
* [feature] Don't emit timestamp in log lines

When running gotosocial with a service manager like systemd, or a
container runtime, the associated log driver usually emits timestamps
itself. In those cases, having the extra timestamp from our own log
lines ends up being a bit noisy and when centrally ingesting logs is
duplicate information.

This introduces a configuration flag that allows disabling emitting the
timestamp. It's only wired up for "daemonised" processes, meaning server
and testrig.

* [chore] Add docs for log-timestamp

* [feature] Simplify timestamp handling

Co-Authored-By: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>

* [chore] Less escaped double-quotes

* [chore] Fix help string

---------

Co-authored-by: kim <89579420+NyaaaWhatsUpDoc@users.noreply.github.com>
2023-08-21 19:07:55 +01:00
tobi
638f023a1c
[performance] Tweak media attachment cleanup; replace stale index (#2143) 2023-08-21 14:03:04 +01:00
dependabot[bot]
70d87f0ff0
[chore]: Bump codeberg.org/gruf/go-kv from 1.6.3 to 1.6.4 (#2142) 2023-08-21 06:54:30 +00:00