all: sync with master; upd chlog

This commit is contained in:
Ainar Garipov 2024-04-02 20:22:19 +03:00
parent ce9bb588ed
commit 6fb2aee210
57 changed files with 1363 additions and 873 deletions

View file

@ -53,9 +53,9 @@
'path': '${{ steps.npm-cache.outputs.dir }}'
'key': "${{ runner.os }}-node-${{ hashFiles('client/package-lock.json') }}"
'restore-keys': '${{ runner.os }}-node-'
- 'name': 'Run make ci'
- 'name': 'Run tests'
'shell': 'bash'
'run': 'make VERBOSE=1 ci'
'run': 'make VERBOSE=1 deps test go-bench go-fuzz'
- 'name': 'Upload coverage'
'uses': 'codecov/codecov-action@v1'
'if': "success() && matrix.os == 'ubuntu-latest'"

View file

@ -14,11 +14,11 @@ and this project adheres to
<!--
## [v0.108.0] - TBA
## [v0.107.47] - 2024-04-03 (APPROX.)
## [v0.107.48] - 2024-04-24 (APPROX.)
See also the [v0.107.47 GitHub milestone][ms-v0.107.47].
See also the [v0.107.48 GitHub milestone][ms-v0.107.48].
[ms-v0.107.47]: https://github.com/AdguardTeam/AdGuardHome/milestone/82?closed=1
[ms-v0.107.48]: https://github.com/AdguardTeam/AdGuardHome/milestone/83?closed=1
NOTE: Add new changes BELOW THIS COMMENT.
-->
@ -29,6 +29,38 @@ NOTE: Add new changes ABOVE THIS COMMENT.
## [v0.107.47] - 2024-04-04
See also the [v0.107.47 GitHub milestone][ms-v0.107.47].
[ms-v0.107.47]: https://github.com/AdguardTeam/AdGuardHome/milestone/82?closed=1
### Changed
- Time Zone Database is now embedded in the binary ([#6758]).
- Failed authentication attempts show the originating IP address in the logs, if
the request came from a trusted proxy ([#5829]).
### Deprecated
- Currently, AdGuard Home uses a best-effort algorithm to fix invalid IDs of
filtering-rule lists on startup. This feature is deprecated, and invalid IDs
will cause errors on startup in a future version.
- Node.JS 16. Future versions will require at least Node.JS 18 to build.
### Fixed
- Resetting DNS upstream mode when applying unrelated settings ([#6851]).
- Symbolic links to config YAML are replaced by a copy of the real file by AGH
after startup ([#6717]).
[#5829]: https://github.com/AdguardTeam/AdGuardHome/issues/5829
[#6717]: https://github.com/AdguardTeam/AdGuardHome/issues/6717
[#6758]: https://github.com/AdguardTeam/AdGuardHome/issues/6758
[#6851]: https://github.com/AdguardTeam/AdGuardHome/issues/6851
## [v0.107.46] - 2024-03-20
See also the [v0.107.46 GitHub milestone][ms-v0.107.46].
@ -42,11 +74,11 @@ See also the [v0.107.46 GitHub milestone][ms-v0.107.46].
### Changed
- Private RDNS resolution (`dns.use_private_ptr_resolvers` in YAML
- Private rDNS resolution (`dns.use_private_ptr_resolvers` in YAML
configuration) now requires a valid "Private reverse DNS servers", when
enabled ([#6820]).
**NOTE:** Disabling private RDNS resolution behaves effectively the same as if
**NOTE:** Disabling private rDNS resolution behaves effectively the same as if
no private reverse DNS servers provided by user and by the OS.
### Fixed
@ -2853,11 +2885,12 @@ See also the [v0.104.2 GitHub milestone][ms-v0.104.2].
<!--
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.47...HEAD
[v0.107.47]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.46...v0.107.46
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.48...HEAD
[v0.107.48]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.47...v0.107.48
-->
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.46...HEAD
[Unreleased]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.47...HEAD
[v0.107.47]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.46...v0.107.47
[v0.107.46]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.45...v0.107.46
[v0.107.45]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.44...v0.107.45
[v0.107.44]: https://github.com/AdguardTeam/AdGuardHome/compare/v0.107.43...v0.107.44

View file

@ -1,89 +1,57 @@
# Contributing to AdGuard Home
# Contributing to AdGuard Home
If you want to contribute to AdGuard Home by filing or commenting on an issue or
opening a pull request, please follow the instructions below.
If you want to contribute to AdGuard Home by filing or commenting on an issue or opening a pull request, please follow the instructions below.
## General recommendations
Please dont:
## General recommendations
- post comments like “+1” or “this”. Use the :+1: reaction on the issue instead, as this allows us to actually see the level of support for issues.
Please don't:
- file issues about localization errors or send localization updates as PRs. Were using [CrowdIn] to manage our translations and we generally update them before each Beta and Release build. You can learn more about translating AdGuard products [in our Knowledge Base][kb-trans].
* post comments like “+1” or “this”. Use the :+1: reaction on the issue
instead, as this allows us to actually see the level of support for issues.
- file issues about a particular filtering-rule list misbehaving. These are tracked through the [separate form for filtering issues][form].
* file issues about localization errors or send localization updates as PRs.
We're using [CrowdIn] to manage our translations and we generally update
them before each Beta and Release build. You can learn more about
translating AdGuard products [in our Knowledge Base][kb-trans].
* file issues about a particular filtering-rule list misbehaving. These are
tracked through the [separate form for filtering issues][form].
* send updates to filtering-rule lists, such as the ones for the Blocked
Services feature or the list of approved filtering-rule lists. We update
them once before each Beta and Release build.
- send or request updates to filtering-rule lists, such as the ones for the Blocked Services feature or the list of approved filtering-rule lists. We update them from the [separate repository][hostlist] once before each Beta and Release build.
Please do:
* follow the template instructions and provide data for reproducing issues.
- follow the template instructions and provide data for reproducing issues.
* write the title of your issue or pull request in English. Any language is
fine in the body, but it is important to keep the title in English to make
it easier for people and bots to look up duplicated issues.
- write the title of your issue or pull request in English. Any language is fine in the body, but it is important to keep the title in English to make it easier for people and bots to look up duplicated issues.
[CrowdIn]: https://crowdin.com/project/adguard-applications/en#/adguard-home
[form]: https://link.adtidy.org/forward.html?action=report&app=home&from=github
[hostlist]: https://github.com/AdguardTeam/HostlistsRegistry
[kb-trans]: https://kb.adguard.com/en/general/adguard-translations
## Issues
### Search first
## Issues
### Search first
Please make sure that the issue is not a duplicate or a question. If it's a
duplicate, please react to the original issue with a thumbs up. If it's a
question, please look through our [Wiki] and, if you haven't found the answer,
post it to the GitHub [Discussions] page.
Please make sure that the issue is not a duplicate or a question. If its a duplicate, please react to the original issue with a thumbs up. If its a question, please look through our [Wiki] and, if you havent found the answer, post it to the GitHub [Discussions] page.
[Discussions]: https://github.com/AdguardTeam/AdGuardHome/discussions/categories/q-a
[Wiki]: https://github.com/AdguardTeam/AdGuardHome/wiki
### Follow the issue template
Developers need to be able to reproduce the faulty behavior in order to fix an issue, so please make sure that you follow the instructions in the issue template carefully.
### Follow the issue template
## Pull requests
Developers need to be able to reproduce the faulty behavior in order to fix an
issue, so please make sure that you follow the instructions in the issue
template carefully.
### Discuss your changes first
Please discuss your changes by opening an issue. The maintainers should evaluate your proposal, and its generally better if thats done before any code is written.
### Review your changes for style
## Pull requests
### Discuss your changes first
Please discuss your changes by opening an issue. The maintainers should
evaluate your proposal, and it's generally better if that's done before any code
is written.
### Review your changes for style
We have a set of [code guidelines][hacking] that we expect the code to follow.
Please make sure you follow it.
We have a set of [code guidelines][hacking] that we expect the code to follow. Please make sure you follow it.
[hacking]: https://github.com/AdguardTeam/CodeGuidelines/blob/master/Go/Go.md
### Test your changes
Make sure that it passes linters and tests by running the corresponding Make targets. For backend changes, its `make go-check`. For frontend, run `make js-lint`.
### Test your changes
Make sure that it passes linters and tests by running the corresponding Make
targets. For backend changes, it's `make go-check`. For frontend, run
`make js-lint`.
Additionally, a manual test is often required. While we're constantly working
on improving our test suites, they're still not as good as we'd like them to be.
Additionally, a manual test is often required. While were constantly working on improving our test suites, theyre still not as good as wed like them to be.

View file

@ -1,65 +1,56 @@
# AdGuard Home Developer Guidelines
# AdGuard Home developer guidelines
This document was moved to the [AdGuard Code Guidelines repository][repo]. All
sections with IDs now only have links to the corresponding files and sections in
that repository.
This document was moved to the [AdGuard Code Guidelines repository][repo]. All sections with IDs now only have links to the corresponding files and sections in that repository.
## <a href="#git" id="git" name="git">Git</a>
## <a href="#git" id="git" name="git">Git</a>
This section was moved to [its own document][git].
## <a href="#go" id="go" name="go">Go</a>
## <a href="#go" id="go" name="go">Go</a>
This section was moved to [its own document][go].
### <a href="#code" id="code" name="code">Code</a>
### <a href="#code" id="code" name="code">Code</a>
This subsection was moved to the [corresponding section][code] of the Go
guidelines document.
This subsection was moved to the [corresponding section][code] of the Go guidelines document.
### <a href="#commenting" id="commenting" name="commenting">Commenting</a>
### <a href="#commenting" id="commenting" name="commenting">Commenting</a>
This subsection was moved to the [corresponding section][cmnt] of the Go
guidelines document.
This subsection was moved to the [corresponding section][cmnt] of the Go guidelines document.
### <a href="#formatting" id="formatting" name="formatting">Formatting</a>
### <a href="#formatting" id="formatting" name="formatting">Formatting</a>
This subsection was moved to the [corresponding section][fmt] of the Go
guidelines document.
This subsection was moved to the [corresponding section][fmt] of the Go guidelines document.
### <a href="#naming" id="naming" name="naming">Naming</a>
### <a href="#naming" id="naming" name="naming">Naming</a>
This subsection was moved to the [corresponding section][name] of the Go
guidelines document.
This subsection was moved to the [corresponding section][name] of the Go guidelines document.
### <a href="#testing" id="testing" name="testing">Testing</a>
### <a href="#testing" id="testing" name="testing">Testing</a>
This subsection was moved to the [corresponding section][test] of the Go
guidelines document.
This subsection was moved to the [corresponding section][test] of the Go guidelines document.
### <a href="#recommended-reading" id="recommended-reading" name="recommended-reading">Recommended Reading</a>
### <a href="#recommended-reading" id="recommended-reading" name="recommended-reading">Recommended Reading</a>
This subsection was moved to the [corresponding section][read] of the Go
guidelines document.
This subsection was moved to the [corresponding section][read] of the Go guidelines document.
## <a href="#markdown" id="markdown" name="markdown">Markdown</a>
## <a href="#markdown" id="markdown" name="markdown">Markdown</a>
This section was moved to [its own document][md].
## <a href="#shell-scripting" id="shell-scripting" name="shell-scripting">Shell Scripting</a>
## <a href="#shell-scripting" id="shell-scripting" name="shell-scripting">Shell Scripting</a>
This section was moved to [its own document][sh].
### <a href="#shell-conditionals" id="shell-conditionals" name="shell-conditionals">Shell Conditionals</a>
### <a href="#shell-conditionals" id="shell-conditionals" name="shell-conditionals">Shell Conditionals</a>
This subsection was moved to the [corresponding section][cond] of the Shell
guidelines document.
This subsection was moved to the [corresponding section][cond] of the Shell guidelines document.
## <a href="#text-including-comments" id="text-including-comments" name="text-including-comments">Text, Including Comments</a>
## <a href="#text-including-comments" id="text-including-comments" name="text-including-comments">Text, Including Comments</a>
This section was moved to [its own document][txt].
## <a href="#yaml" id="yaml" name="yaml">YAML</a>
## <a href="#yaml" id="yaml" name="yaml">YAML</a>
This section was moved to [its own document][yaml].

View file

@ -82,8 +82,6 @@ build: deps quick-build
quick-build: js-build go-build
ci: deps test go-bench go-fuzz
deps: js-deps go-deps
lint: js-lint go-lint
test: js-test go-test
@ -98,15 +96,10 @@ build-release: $(BUILD_RELEASE_DEPS_$(FRONTEND_PREBUILT))
clean: ; $(ENV) "$(SHELL)" ./scripts/make/clean.sh
init: ; git config core.hooksPath ./scripts/hooks
js-build:
$(NPM) $(NPM_FLAGS) run build-prod
js-deps:
$(NPM) $(NPM_INSTALL_FLAGS) ci
# TODO(a.garipov): Remove the legacy client tasks support once the new
# client is done and the old one is removed.
js-lint: ; $(NPM) $(NPM_FLAGS) run lint
js-test: ; $(NPM) $(NPM_FLAGS) run test
js-build: ; $(NPM) $(NPM_FLAGS) run build-prod
js-deps: ; $(NPM) $(NPM_INSTALL_FLAGS) ci
js-lint: ; $(NPM) $(NPM_FLAGS) run lint
js-test: ; $(NPM) $(NPM_FLAGS) run test
go-bench: ; $(ENV) "$(SHELL)" ./scripts/make/go-bench.sh
go-build: ; $(ENV) "$(SHELL)" ./scripts/make/go-build.sh

474
README.md
View file

@ -1,85 +1,75 @@
&nbsp;
<p align="center">
<picture>
<source media="(prefers-color-scheme: dark)" srcset="doc/adguard_home_darkmode.svg">
<img alt="AdGuard Home" src="doc/adguard_home_lightmode.svg" width="300px">
</picture>
<picture>
<source media="(prefers-color-scheme: dark)" srcset="doc/adguard_home_darkmode.svg">
<img alt="AdGuard Home" src="doc/adguard_home_lightmode.svg" width="300px">
</picture>
</p>
<h3 align="center">Privacy protection center for you and your devices</h3>
<p align="center">
Free and open source, powerful network-wide ads & trackers blocking DNS
server.
Free and open source, powerful network-wide ads & trackers blocking DNS server.
</p>
<p align="center">
<a href="https://adguard.com/">AdGuard.com</a> |
<a href="https://github.com/AdguardTeam/AdGuardHome/wiki">Wiki</a> |
<a href="https://reddit.com/r/Adguard">Reddit</a> |
<a href="https://twitter.com/AdGuard">Twitter</a> |
<a href="https://t.me/adguard_en">Telegram</a>
<br/><br/>
<a href="https://codecov.io/github/AdguardTeam/AdGuardHome?branch=master">
<img src="https://img.shields.io/codecov/c/github/AdguardTeam/AdGuardHome/master.svg" alt="Code Coverage"/>
</a>
<a href="https://goreportcard.com/report/AdguardTeam/AdGuardHome">
<img src="https://goreportcard.com/badge/github.com/AdguardTeam/AdGuardHome" alt="Go Report Card"/>
</a>
<a href="https://hub.docker.com/r/adguard/adguardhome">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/adguard/adguardhome.svg?maxAge=604800"/>
</a>
<br/>
<a href="https://github.com/AdguardTeam/AdGuardHome/releases">
<img src="https://img.shields.io/github/release/AdguardTeam/AdGuardHome/all.svg" alt="Latest release"/>
</a>
<a href="https://snapcraft.io/adguard-home">
<img alt="adguard-home" src="https://snapcraft.io/adguard-home/badge.svg"/>
</a>
<a href="https://adguard.com/">AdGuard.com</a> |
<a href="https://github.com/AdguardTeam/AdGuardHome/wiki">Wiki</a> |
<a href="https://reddit.com/r/Adguard">Reddit</a> |
<a href="https://twitter.com/AdGuard">Twitter</a> |
<a href="https://t.me/adguard_en">Telegram</a>
<br/><br/>
<a href="https://codecov.io/github/AdguardTeam/AdGuardHome?branch=master">
<img src="https://img.shields.io/codecov/c/github/AdguardTeam/AdGuardHome/master.svg" alt="Code Coverage"/>
</a>
<a href="https://goreportcard.com/report/AdguardTeam/AdGuardHome">
<img src="https://goreportcard.com/badge/github.com/AdguardTeam/AdGuardHome" alt="Go Report Card"/>
</a>
<a href="https://hub.docker.com/r/adguard/adguardhome">
<img alt="Docker Pulls" src="https://img.shields.io/docker/pulls/adguard/adguardhome.svg?maxAge=604800"/>
</a>
<br/>
<a href="https://github.com/AdguardTeam/AdGuardHome/releases">
<img src="https://img.shields.io/github/release/AdguardTeam/AdGuardHome/all.svg" alt="Latest release"/>
</a>
<a href="https://snapcraft.io/adguard-home">
<img alt="adguard-home" src="https://snapcraft.io/adguard-home/badge.svg"/>
</a>
</p>
<br/>
<p align="center">
<img src="https://cdn.adtidy.org/public/Adguard/Common/adguard_home.gif" width="800"/>
<img src="https://cdn.adtidy.org/public/Adguard/Common/adguard_home.gif" width="800"/>
</p>
<hr/>
AdGuard Home is a network-wide software for blocking ads and tracking. After you
set it up, it'll cover ALL your home devices, and you don't need any client-side
software for that.
AdGuard Home is a network-wide software for blocking ads and tracking. After you set it up, it'll cover ALL your home devices, and you don't need any client-side software for that.
It operates as a DNS server that re-routes tracking domains to a “black hole”,
thus preventing your devices from connecting to those servers. It's based on
software we use for our public [AdGuard DNS] servers, and both share a lot of
code.
It operates as a DNS server that re-routes tracking domains to a “black hole”, thus preventing your devices from connecting to those servers. It's based on software we use for our public [AdGuard DNS] servers, and both share a lot of code.
[AdGuard DNS]: https://adguard-dns.io/
- [Getting Started](#getting-started)
- [Automated install (Linux/Unix/MacOS/FreeBSD/OpenBSD)](#automated-install-linux-and-mac)
- [Alternative methods](#alternative-methods)
- [Guides](#guides)
- [API](#api)
- [Comparing AdGuard Home to other solutions](#comparison)
- [How is this different from public AdGuard DNS servers?](#comparison-adguard-dns)
- [How does AdGuard Home compare to Pi-Hole](#comparison-pi-hole)
- [How does AdGuard Home compare to traditional ad blockers](#comparison-adblock)
- [Known limitations](#comparison-limitations)
- [How to build from source](#how-to-build)
- [Prerequisites](#prerequisites)
- [Building](#building)
- [Contributing](#contributing)
- [Test unstable versions](#test-unstable-versions)
- [Reporting issues](#reporting-issues)
- [Help with translations](#translate)
- [Other](#help-other)
- [Projects that use AdGuard Home](#uses)
- [Acknowledgments](#acknowledgments)
- [Privacy](#privacy)
## <a href="#getting-started" id="getting-started" name="getting-started">Getting Started</a>
* [Getting Started](#getting-started)
* [Automated install (Linux/Unix/MacOS/FreeBSD/OpenBSD)](#automated-install-linux-and-mac)
* [Alternative methods](#alternative-methods)
* [Guides](#guides)
* [API](#api)
* [Comparing AdGuard Home to other solutions](#comparison)
* [How is this different from public AdGuard DNS servers?](#comparison-adguard-dns)
* [How does AdGuard Home compare to Pi-Hole](#comparison-pi-hole)
* [How does AdGuard Home compare to traditional ad blockers](#comparison-adblock)
* [Known limitations](#comparison-limitations)
* [How to build from source](#how-to-build)
* [Prerequisites](#prerequisites)
* [Building](#building)
* [Contributing](#contributing)
* [Test unstable versions](#test-unstable-versions)
* [Reporting issues](#reporting-issues)
* [Help with translations](#translate)
* [Other](#help-other)
* [Projects that use AdGuard Home](#uses)
* [Acknowledgments](#acknowledgments)
* [Privacy](#privacy)
## <a href="#getting-started" id="getting-started" name="getting-started">Getting Started</a>
### <a href="#automated-install-linux-and-mac" id="automated-install-linux-and-mac" name="automated-install-linux-and-mac">Automated install (Linux/Unix/MacOS/FreeBSD/OpenBSD)</a>
### <a href="#automated-install-linux-and-mac" id="automated-install-linux-and-mac" name="automated-install-linux-and-mac">Automated install (Linux/Unix/MacOS/FreeBSD/OpenBSD)</a>
To install with `curl` run the following command:
@ -101,95 +91,70 @@ fetch -o - https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scri
The script also accepts some options:
* `-c <channel>` to use specified channel;
* `-r` to reinstall AdGuard Home;
* `-u` to uninstall AdGuard Home;
* `-v` for verbose output.
- `-c <channel>` to use specified channel;
- `-r` to reinstall AdGuard Home;
- `-u` to uninstall AdGuard Home;
- `-v` for verbose output.
Note that options `-r` and `-u` are mutually exclusive.
### <a href="#alternative-methods" id="alternative-methods" name="alternative-methods">Alternative methods</a>
#### <a href="#manual-installation" id="manual-installation" name="manual-installation">Manual installation</a>
### <a href="#alternative-methods" id="alternative-methods" name="alternative-methods">Alternative methods</a>
Please read the **[Getting Started][wiki-start]** article on our Wiki to learn how to install AdGuard Home manually, and how to configure your devices to use it.
#### <a href="#manual-installation" id="manual-installation" name="manual-installation">Manual installation</a>
Please read the **[Getting Started][wiki-start]** article on our Wiki to learn
how to install AdGuard Home manually, and how to configure your devices to use
it.
#### <a href="#docker" id="docker" name="docker">Docker</a>
#### <a href="#docker" id="docker" name="docker">Docker</a>
You can use our official Docker image on [Docker Hub].
#### <a href="#snap-store" id="snap-store" name="snap-store">Snap Store</a>
#### <a href="#snap-store" id="snap-store" name="snap-store">Snap Store</a>
If you're running **Linux,** there's a secure and easy way to install AdGuard
Home: get it from the [Snap Store].
If you're running **Linux,** there's a secure and easy way to install AdGuard Home: get it from the [Snap Store].
[Docker Hub]: https://hub.docker.com/r/adguard/adguardhome
[Snap Store]: https://snapcraft.io/adguard-home
[wiki-start]: https://github.com/AdguardTeam/AdGuardHome/wiki/Getting-Started
### <a href="#guides" id="guides" name="guides">Guides</a>
### <a href="#guides" id="guides" name="guides">Guides</a>
See our [Wiki][wiki].
[wiki]: https://github.com/AdguardTeam/AdGuardHome/wiki
### <a href="#api" id="api" name="api">API</a>
### <a href="#api" id="api" name="api">API</a>
If you want to integrate with AdGuard Home, you can use our [REST API][openapi].
Alternatively, you can use this [python client][pyclient], which is used to
build the [AdGuard Home Hass.io Add-on][hassio].
If you want to integrate with AdGuard Home, you can use our [REST API][openapi]. Alternatively, you can use this [python client][pyclient], which is used to build the [AdGuard Home Hass.io Add-on][hassio].
[hassio]: https://www.home-assistant.io/integrations/adguard/
[openapi]: https://github.com/AdguardTeam/AdGuardHome/tree/master/openapi
[pyclient]: https://pypi.org/project/adguardhome/
## <a href="#comparison" id="comparison" name="comparison">Comparing AdGuard Home to other solutions</a>
### <a href="#comparison-adguard-dns" id="comparison-adguard-dns" name="comparison-adguard-dns">How is this different from public AdGuard DNS servers?</a>
## <a href="#comparison" id="comparison" name="comparison">Comparing AdGuard Home to other solutions</a>
Running your own AdGuard Home server allows you to do much more than using a public DNS server. It's a completely different level. See for yourself:
### <a href="#comparison-adguard-dns" id="comparison-adguard-dns" name="comparison-adguard-dns">How is this different from public AdGuard DNS servers?</a>
- Choose what exactly the server blocks and permits.
Running your own AdGuard Home server allows you to do much more than using a
public DNS server. It's a completely different level. See for yourself:
- Monitor your network activity.
* Choose what exactly the server blocks and permits.
- Add your own custom filtering rules.
* Monitor your network activity.
- **Most importantly, it's your own server, and you are the only one who's in control.**
* Add your own custom filtering rules.
### <a href="#comparison-pi-hole" id="comparison-pi-hole" name="comparison-pi-hole">How does AdGuard Home compare to Pi-Hole</a>
* **Most importantly, it's your own server, and you are the only one who's in
control.**
At this point, AdGuard Home has a lot in common with Pi-Hole. Both block ads and trackers using the so-called “DNS sinkholing” method and both allow customizing what's blocked.
> [!NOTE]
> We're not going to stop here. DNS sinkholing is not a bad starting point, but this is just the beginning.
AdGuard Home provides a lot of features out-of-the-box with no need to install and configure additional software. We want it to be simple to the point when even casual users can set it up with minimal effort.
### <a href="#comparison-pi-hole" id="comparison-pi-hole" name="comparison-pi-hole">How does AdGuard Home compare to Pi-Hole</a>
At this point, AdGuard Home has a lot in common with Pi-Hole. Both block ads
and trackers using the so-called “DNS sinkholing” method and both allow
customizing what's blocked.
<aside>
We're not going to stop here. DNS sinkholing is not a bad starting point, but
this is just the beginning.
</aside>
AdGuard Home provides a lot of features out-of-the-box with no need to install
and configure additional software. We want it to be simple to the point when
even casual users can set it up with minimal effort.
**Disclaimer:** some of the listed features can be added to Pi-Hole by
installing additional software or by manually using SSH terminal and
reconfiguring one of the utilities Pi-Hole consists of. However, in our
opinion, this cannot be legitimately counted as a Pi-Hole's feature.
> [!NOTE]
> Some of the listed features can be added to Pi-Hole by installing additional software or by manually using SSH terminal and reconfiguring one of the utilities Pi-Hole consists of. However, in our opinion, this cannot be legitimately counted as a Pi-Hole's feature.
| Feature | AdGuard&nbsp;Home | Pi-Hole |
|-------------------------------------------------------------------------|-------------------|-----------------------------------------------------------|
@ -207,68 +172,45 @@ opinion, this cannot be legitimately counted as a Pi-Hole's feature.
| Access settings (choose who can use AGH DNS) | ✅ | ❌ |
| Running [without root privileges][wiki-noroot] | ✅ | ❌ |
[wiki-noroot]: https://github.com/AdguardTeam/AdGuardHome/wiki/Getting-Started#running-without-superuser
[wiki-noroot]: https://adguard-dns.io/kb/adguard-home/getting-started/#running-without-superuser
### <a href="#comparison-adblock" id="comparison-adblock" name="comparison-adblock">How does AdGuard Home compare to traditional ad blockers</a>
### <a href="#comparison-adblock" id="comparison-adblock" name="comparison-adblock">How does AdGuard Home compare to traditional ad blockers</a>
It depends.
DNS sinkholing is capable of blocking a big percentage of ads, but it lacks
the flexibility and the power of traditional ad blockers. You can get a good
impression about the difference between these methods by reading [this
article][blog-adaway], which compares AdGuard for Android (a traditional ad
blocker) to hosts-level ad blockers (which are almost identical to DNS-based
blockers in their capabilities). This level of protection is enough for some
users.
DNS sinkholing is capable of blocking a big percentage of ads, but it lacks the flexibility and the power of traditional ad blockers. You can get a good impression about the difference between these methods by reading [this article][blog-adaway], which compares AdGuard for Android (a traditional ad blocker) to hosts-level ad blockers (which are almost identical to DNS-based blockers in their capabilities). This level of protection is enough for some users.
Additionally, using a DNS-based blocker can help to block ads, tracking and
analytics requests on other types of devices, such as SmartTVs, smart speakers
or other kinds of IoT devices (on which you can't install traditional ad
blockers).
Additionally, using a DNS-based blocker can help to block ads, tracking and analytics requests on other types of devices, such as SmartTVs, smart speakers or other kinds of IoT devices (on which you can't install traditional ad blockers).
### <a href="#comparison-limitations" id="comparison-limitations" name="comparison-limitations">Known limitations</a>
### <a href="#comparison-limitations" id="comparison-limitations" name="comparison-limitations">Known limitations</a>
Here are some examples of what cannot be blocked by a DNS-level blocker:
* YouTube, Twitch ads;
- YouTube, Twitch ads;
* Facebook, Twitter, Instagram sponsored posts.
- Facebook, Twitter, Instagram sponsored posts.
Essentially, any advertising that shares a domain with content cannot be blocked
by a DNS-level blocker.
Essentially, any advertising that shares a domain with content cannot be blocked by a DNS-level blocker.
Is there a chance to handle this in the future? DNS will never be enough to do
this. Our only option is to use a content blocking proxy like what we do in the
standalone AdGuard applications. We're [going to bring][issue-1228] this
feature support to AdGuard Home in the future. Unfortunately, even in this
case, there still will be cases when this won't be enough or would require quite
a complicated configuration.
Is there a chance to handle this in the future? DNS will never be enough to do this. Our only option is to use a content blocking proxy like what we do in the standalone AdGuard applications. We're [going to bring][issue-1228] this feature support to AdGuard Home in the future. Unfortunately, even in this case, there still will be cases when this won't be enough or would require quite a complicated configuration.
[blog-adaway]: https://adguard.com/blog/adguard-vs-adaway-dns66.html
[issue-1228]: https://github.com/AdguardTeam/AdGuardHome/issues/1228
## <a href="#how-to-build" id="how-to-build" name="how-to-build">How to build from source</a>
## <a href="#how-to-build" id="how-to-build" name="how-to-build">How to build from source</a>
### <a href="#prerequisites" id="prerequisites" name="prerequisites">Prerequisites</a>
### <a href="#prerequisites" id="prerequisites" name="prerequisites">Prerequisites</a>
Run `make init` to prepare the development environment.
You will need this to build AdGuard Home:
* [Go](https://golang.org/dl/) v1.20 or later;
* [Node.js](https://nodejs.org/en/download/) v16 or later;
* [npm](https://www.npmjs.com/) v8 or later;
* [yarn](https://yarnpkg.com/) v1.22.5 or later.
- [Go](https://golang.org/dl/) v1.20 or later;
- [Node.js](https://nodejs.org/en/download/) v16 or later;
- [npm](https://www.npmjs.com/) v8 or later;
- [yarn](https://yarnpkg.com/) v1.22.5 or later.
### <a href="#building" id="building" name="building">Building</a>
### <a href="#building" id="building" name="building">Building</a>
Open your terminal and execute these commands:
@ -278,27 +220,22 @@ cd AdGuardHome
make
```
#### <a href="#building-node" id="building-node" name="building-node">Building with Node.js 17 and later</a>
#### <a href="#building-node" id="building-node" name="building-node">Building with Node.js 17 and later</a>
In order to build AdGuard Home with Node.js 17 and later, specify
`--openssl-legacy-provider` option.
In order to build AdGuard Home with Node.js 17 and later, specify `--openssl-legacy-provider` option.
```sh
export NODE_OPTIONS=--openssl-legacy-provider
```
**NOTE:** The non-standard `-j` flag is currently not supported, so building
with `make -j 4` or setting your `MAKEFLAGS` to include, for example, `-j 4` is
likely to break the build. If you do have your `MAKEFLAGS` set to that, and you
don't want to change it, you can override it by running `make -j 1`.
> [!WARNING]
> The non-standard `-j` flag is currently not supported, so building with `make -j 4` or setting your `MAKEFLAGS` to include, for example, `-j 4` is likely to break the build. If you do have your `MAKEFLAGS` set to that, and you don't want to change it, you can override it by running `make -j 1`.
Check the [`Makefile`][src-makefile] to learn about other commands.
#### <a href="#building-cross" id="building-cross" name="building-cross">Building for a different platform</a>
#### <a href="#building-cross" id="building-cross" name="building-cross">Building for a different platform</a>
You can build AdGuard Home for any OS/ARCH that Go supports. In order to do
this, specify `GOOS` and `GOARCH` environment variables as macros when running
`make`.
You can build AdGuard Home for any OS/ARCH that Go supports. In order to do this, specify `GOOS` and `GOARCH` environment variables as macros when running `make`.
For example:
@ -312,10 +249,9 @@ or:
make GOOS='linux' GOARCH='arm64'
```
#### <a href="#preparing-releases" id="preparing-releases" name="preparing-releases">Preparing releases</a>
#### <a href="#preparing-releases" id="preparing-releases" name="preparing-releases">Preparing releases</a>
You'll need [`snapcraft`] to prepare a release build. Once installed, run the
following command:
You'll need [`snapcraft`] to prepare a release build. Once installed, run the following command:
```sh
make build-release CHANNEL='...' VERSION='...'
@ -323,47 +259,39 @@ make build-release CHANNEL='...' VERSION='...'
See the [`build-release` target documentation][targ-release].
#### <a href="#docker-image" id="docker-image" name="docker-image">Docker image</a>
#### <a href="#docker-image" id="docker-image" name="docker-image">Docker image</a>
Run `make build-docker` to build the Docker image locally (the one that we
publish to DockerHub). Please note, that we're using [Docker Buildx][buildx] to
build our official image.
Run `make build-docker` to build the Docker image locally (the one that we publish to DockerHub). Please note, that we're using [Docker Buildx][buildx] to build our official image.
You may need to prepare before using these builds:
* (Linux-only) Install Qemu:
- (Linux-only) Install Qemu:
```sh
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes
```
```sh
docker run --rm --privileged multiarch/qemu-user-static --reset -p yes --credential yes
```
* Prepare the builder:
- Prepare the builder:
```sh
docker buildx create --name buildx-builder --driver docker-container --use
```
```sh
docker buildx create --name buildx-builder --driver docker-container --use
```
See the [`build-docker` target documentation][targ-docker].
#### <a href="#debugging-the-frontend" id="debugging-the-frontend" name="debugging-the-frontend">Debugging the frontend</a>
#### <a href="#debugging-the-frontend" id="debugging-the-frontend" name="debugging-the-frontend">Debugging the frontend</a>
When you need to debug the frontend without recompiling the production version
every time, for example to check how your labels would look on a form, you can
run the frontend build a development environment.
When you need to debug the frontend without recompiling the production version every time, for example to check how your labels would look on a form, you can run the frontend build a development environment.
1. In a separate terminal, run:
1. In a separate terminal, run:
```sh
( cd ./client/ && env NODE_ENV='development' npm run watch )
```
```sh
( cd ./client/ && env NODE_ENV='development' npm run watch )
```
2. Run your `AdGuardHome` binary with the `--local-frontend` flag, which
instructs AdGuard Home to ignore the built-in frontend files and use those
from the `./build/` directory.
2. Run your `AdGuardHome` binary with the `--local-frontend` flag, which instructs AdGuard Home to ignore the built-in frontend files and use those from the `./build/` directory.
3. Now any changes you make in the `./client/` directory should be recompiled
and become available on the web UI. Make sure that you disable the browser
cache to make sure that you actually get the recompiled version.
3. Now any changes you make in the `./client/` directory should be recompiled and become available on the web UI. Make sure that you disable the browser cache to make sure that you actually get the recompiled version.
[`snapcraft`]: https://snapcraft.io/
[buildx]: https://docs.docker.com/buildx/working-with-buildx/
@ -371,170 +299,120 @@ run the frontend build a development environment.
[targ-docker]: https://github.com/AdguardTeam/AdGuardHome/tree/master/scripts#build-dockersh-build-a-multi-architecture-docker-image
[targ-release]: https://github.com/AdguardTeam/AdGuardHome/tree/master/scripts#build-releasesh-build-a-release-for-all-platforms
## <a href="#contributing" id="contributing" name="contributing">Contributing</a>
You are welcome to fork this repository, make your changes and [submit a pull
request][pr]. Please make sure you follow our [code guidelines][guide] though.
You are welcome to fork this repository, make your changes and [submit a pull request][pr]. Please make sure you follow our [code guidelines][guide] though.
Please note that we don't expect people to contribute to both UI and backend
parts of the program simultaneously. Ideally, the backend part is implemented
first, i.e. configuration, API, and the functionality itself. The UI part can
be implemented later in a different pull request by a different person.
Please note that we don't expect people to contribute to both UI and backend parts of the program simultaneously. Ideally, the backend part is implemented first, i.e. configuration, API, and the functionality itself. The UI part can be implemented later in a different pull request by a different person.
[guide]: https://github.com/AdguardTeam/CodeGuidelines/
[pr]: https://github.com/AdguardTeam/AdGuardHome/pulls
### <a href="#test-unstable-versions" id="test-unstable-versions" name="test-unstable-versions">Test unstable versions</a>
### <a href="#test-unstable-versions" id="test-unstable-versions" name="test-unstable-versions">Test unstable versions</a>
There are two update channels that you can use:
* `beta`: beta versions of AdGuard Home. More or less stable versions,
usually released every two weeks or more often.
- `beta`: beta versions of AdGuard Home. More or less stable versions, usually released every two weeks or more often.
* `edge`: the newest version of AdGuard Home from the development branch. New
updates are pushed to this channel daily.
- `edge`: the newest version of AdGuard Home from the development branch. New updates are pushed to this channel daily.
There are three options how you can install an unstable version:
1. [Snap Store]: look for the `beta` and `edge` channels.
1. [Snap Store]: look for the `beta` and `edge` channels.
2. [Docker Hub]: look for the `beta` and `edge` tags.
2. [Docker Hub]: look for the `beta` and `edge` tags.
3. Standalone builds. Use the automated installation script or look for the
available builds [on the Wiki][wiki-platf].
3. Standalone builds. Use the automated installation script or look for the available builds [on the Wiki][wiki-platf].
Script to install a beta version:
Script to install a beta version:
```sh
curl -s -S -L https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scripts/install.sh | sh -s -- -c beta
```
```sh
curl -s -S -L https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scripts/install.sh | sh -s -- -c beta
```
Script to install an edge version:
Script to install an edge version:
```sh
curl -s -S -L https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scripts/install.sh | sh -s -- -c edge
```
```sh
curl -s -S -L https://raw.githubusercontent.com/AdguardTeam/AdGuardHome/master/scripts/install.sh | sh -s -- -c edge
```
[wiki-platf]: https://github.com/AdguardTeam/AdGuardHome/wiki/Platforms
### <a href="#reporting-issues" id="reporting-issues" name="reporting-issues">Report issues</a>
### <a href="#reporting-issues" id="reporting-issues" name="reporting-issues">Report issues</a>
If you run into any problem or have a suggestion, head to [this page][iss] and
click on the “New issue” button. Please follow the instructions in the issue
form carefully and don't forget to start by searching for duplicates.
If you run into any problem or have a suggestion, head to [this page][iss] and click on the “New issue” button. Please follow the instructions in the issue form carefully and don't forget to start by searching for duplicates.
[iss]: https://github.com/AdguardTeam/AdGuardHome/issues
### <a href="#translate" id="translate" name="translate">Help with translations</a>
### <a href="#translate" id="translate" name="translate">Help with translations</a>
If you want to help with AdGuard Home translations, please learn more about
translating AdGuard products [in our Knowledge Base][kb-trans]. You can
contribute to the [AdGuardHome project on CrowdIn][crowdin].
If you want to help with AdGuard Home translations, please learn more about translating AdGuard products [in our Knowledge Base][kb-trans]. You can contribute to the [AdGuardHome project on CrowdIn][crowdin].
[crowdin]: https://crowdin.com/project/adguard-applications/en#/adguard-home
[kb-trans]: https://kb.adguard.com/en/general/adguard-translations
### <a href="#help-other" id="help-other" name="help-other">Other</a>
### <a href="#help-other" id="help-other" name="help-other">Other</a>
Another way you can contribute is by [looking for issues][iss-help] marked as
`help wanted`, asking if the issue is up for grabs, and sending a PR fixing the
bug or implementing the feature.
Another way you can contribute is by [looking for issues][iss-help] marked as `help wanted`, asking if the issue is up for grabs, and sending a PR fixing the bug or implementing the feature.
[iss-help]: https://github.com/AdguardTeam/AdGuardHome/issues?q=is%3Aissue+is%3Aopen+label%3A%22help+wanted%22
## <a href="#uses" id="uses" name="uses">Projects that use AdGuard Home</a>
<!--
TODO(a.garipov): Use reference links.
-->
Please note that these projects are not affiliated with AdGuard, but are made by third-party developers and fans.
* [AdGuard Home Remote](https://apps.apple.com/app/apple-store/id1543143740):
iOS app by [Joost](https://rocketscience-it.nl/).
- [AdGuard Home Remote](https://apps.apple.com/app/apple-store/id1543143740): iOS app by [Joost](https://rocketscience-it.nl/).
* [Python library](https://github.com/frenck/python-adguardhome) by
[@frenck](https://github.com/frenck).
- [Python library](https://github.com/frenck/python-adguardhome) by [@frenck](https://github.com/frenck).
* [Home Assistant add-on](https://github.com/hassio-addons/addon-adguard-home)
by [@frenck](https://github.com/frenck).
- [Home Assistant add-on](https://github.com/hassio-addons/addon-adguard-home) by [@frenck](https://github.com/frenck).
* [OpenWrt LUCI app](https://github.com/kongfl888/luci-app-adguardhome) by
[@kongfl888](https://github.com/kongfl888) (originally by
[@rufengsuixing](https://github.com/rufengsuixing)).
- [OpenWrt LUCI app](https://github.com/kongfl888/luci-app-adguardhome) by [@kongfl888](https://github.com/kongfl888) (originally by [@rufengsuixing](https://github.com/rufengsuixing)).
* [AdGuardHome sync](https://github.com/bakito/adguardhome-sync) by
[@bakito](https://github.com/bakito).
- [AdGuardHome sync](https://github.com/bakito/adguardhome-sync) by [@bakito](https://github.com/bakito).
* [Terminal-based, real-time traffic monitoring and statistics for your AdGuard Home
instance](https://github.com/Lissy93/AdGuardian-Term) by
[@Lissy93](https://github.com/Lissy93)
- [Terminal-based, real-time traffic monitoring and statistics for your AdGuard Home instance](https://github.com/Lissy93/AdGuardian-Term) by [@Lissy93](https://github.com/Lissy93)
* [AdGuard Home on GLInet
routers](https://forum.gl-inet.com/t/adguardhome-on-gl-routers/10664) by
[Gl-Inet](https://gl-inet.com/).
- [AdGuard Home on GLInet routers](https://forum.gl-inet.com/t/adguardhome-on-gl-routers/10664) by [Gl-Inet](https://gl-inet.com/).
* [Cloudron app](https://git.cloudron.io/cloudron/adguard-home-app) by
[@gramakri](https://github.com/gramakri).
- [Cloudron app](https://git.cloudron.io/cloudron/adguard-home-app) by [@gramakri](https://github.com/gramakri).
* [Asuswrt-Merlin-AdGuardHome-Installer](https://github.com/jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer)
by [@jumpsmm7](https://github.com/jumpsmm7) aka
[@SomeWhereOverTheRainBow](https://www.snbforums.com/members/somewhereovertherainbow.64179/).
- [Asuswrt-Merlin-AdGuardHome-Installer](https://github.com/jumpsmm7/Asuswrt-Merlin-AdGuardHome-Installer) by [@jumpsmm7](https://github.com/jumpsmm7) aka [@SomeWhereOverTheRainBow](https://www.snbforums.com/members/somewhereovertherainbow.64179/).
* [Node.js library](https://github.com/Andrea055/AdguardHomeAPI) by
[@Andrea055](https://github.com/Andrea055/).
- [Node.js library](https://github.com/Andrea055/AdguardHomeAPI) by [@Andrea055](https://github.com/Andrea055/).
* [Browser Extension](https://github.com/satheshshiva/Adguard-Home-Browser-Ext) by
[@satheshshiva](https://github.com/satheshshiva/).
- [Browser Extension](https://github.com/satheshshiva/Adguard-Home-Browser-Ext) by [@satheshshiva](https://github.com/satheshshiva/).
- [Zabbix Template for AdGuard Home](https://github.com/diasdmhub/AdGuard_Home_Zabbix_Template) by [@diasdmhub](https://github.com/diasdmhub).
- [Chocolatey package](https://community.chocolatey.org/packages/adguardhome/) by [niks255](https://community.chocolatey.org/profiles/niks255).
## <a href="#acknowledgments" id="acknowledgments" name="acknowledgments">Acknowledgments</a>
<!--
TODO(a.garipov): Use reference links.
-->
This software wouldn't have been possible without:
* [Go](https://golang.org/dl/) and its libraries:
* [gcache](https://github.com/bluele/gcache)
* [miekg's dns](https://github.com/miekg/dns)
* [go-yaml](https://github.com/go-yaml/yaml)
* [service](https://godoc.org/github.com/kardianos/service)
* [dnsproxy](https://github.com/AdguardTeam/dnsproxy)
* [urlfilter](https://github.com/AdguardTeam/urlfilter)
* [Node.js](https://nodejs.org/) and its libraries:
* And many more Node.js packages.
* [React.js](https://reactjs.org)
* [Tabler](https://github.com/tabler/tabler)
* [whotracks.me data](https://github.com/cliqz-oss/whotracks.me)
- [Go](https://golang.org/dl/) and its libraries:
- [gcache](https://github.com/bluele/gcache)
- [miekg's dns](https://github.com/miekg/dns)
- [go-yaml](https://github.com/go-yaml/yaml)
- [service](https://godoc.org/github.com/kardianos/service)
- [dnsproxy](https://github.com/AdguardTeam/dnsproxy)
- [urlfilter](https://github.com/AdguardTeam/urlfilter)
- [Node.js](https://nodejs.org/) and its libraries:
- [React.js](https://reactjs.org)
- [Tabler](https://github.com/tabler/tabler)
- And many more Node.js packages.
- [whotracks.me data](https://github.com/cliqz-oss/whotracks.me)
You might have seen that [CoreDNS] was mentioned here before, but we've stopped
using it in AdGuard Home.
You might have seen that [CoreDNS] was mentioned here before, but we've stopped using it in AdGuard Home.
For the full list of all Node.js packages in use, please take a look at
[`client/package.json`][src-packagejson] file.
For the full list of all Node.js packages in use, please take a look at [`client/package.json`][src-packagejson] file.
[CoreDNS]: https://coredns.io
[src-packagejson]: https://github.com/AdguardTeam/AdGuardHome/blob/master/client/package.json
## <a href="#privacy" id="privacy" name="privacy">Privacy</a>
## <a href="#privacy" id="privacy" name="privacy">Privacy</a>
Our main idea is that you are the one, who should be in control of your data.
So it is only natural, that AdGuard Home does not collect any usage statistics,
and does not use any web services unless you configure it to do so. See also
the [full privacy policy][privacy] with every bit that *could in theory be sent*
by AdGuard Home is available.
Our main idea is that you are the one, who should be in control of your data. So it is only natural, that AdGuard Home does not collect any usage statistics, and does not use any web services unless you configure it to do so. See also the [full privacy policy][privacy] with every bit that *could in theory be sent* by AdGuard Home is available.
[privacy]: https://adguard.com/en/privacy/home.html

View file

@ -1,18 +1,13 @@
# Security Policy
# Security Policy
## Reporting a Vulnerability
## Reporting vulnerabilities
Please send your vulnerability reports to <security@adguard.com>. To make sure
that your report reaches us, please:
Please send your vulnerability reports to <security@adguard.com>. To make sure that your report reaches us, please:
1. Include the words “AdGuard Home” and “vulnerability” to the subject line as
well as a short description of the vulnerability. For example:
1. Include the words “AdGuard Home” and “vulnerability” to the subject line as well as a short description of the vulnerability. For example:
> AdGuard Home API vulnerability: possible XSS attack
> AdGuard Home API vulnerability: possible XSS attack
2. Make sure that the message body contains a clear description of the
vulnerability.
1. Make sure that the message body contains a clear description of the vulnerability.
If you have not received a reply to your email within 7 days, please make sure
to follow up with us again at <security@adguard.com>. Once again, make sure
that the word “vulnerability” is in the subject line.
If you have not received a reply to your email within 7 days, please make sure to follow up with us again at <security@adguard.com>. Once again, make sure that the word “vulnerability” is in the subject line.

View file

@ -7,7 +7,8 @@
# Make sure to sync any changes with the branch overrides below.
'variables':
'channel': 'edge'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerFrontend': 'adguard/home-js-builder:1.1'
'dockerGo': 'adguard/go-builder:1.21.8--1'
'stages':
- 'Build frontend':
@ -40,11 +41,14 @@
'jobs':
- 'Publish to GitHub Releases'
# TODO(e.burkov): In jobs below find out why the explicit checkout is
# performed.
'Build frontend':
'artifacts':
- 'name': 'AdGuardHome frontend'
'pattern': 'build/**'
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerGo}'
'image': '${bamboo.dockerFrontend}'
'volumes':
'${system.YARN_DIR}': '${bamboo.cacheYarn}'
'key': 'BF'
@ -61,19 +65,21 @@
set -e -f -u -x
# Explicitly checkout the revision that we need.
git checkout "${bamboo.repository.revision.number}"
make js-deps js-build
'artifacts':
- 'name': 'AdGuardHome frontend'
'pattern': 'build/**'
'shared': true
'required': true
make\
VERBOSE=1\
js-deps js-build
'requirements':
- 'adg-docker': 'true'
'Make release':
'artifact-subscriptions':
- 'artifact': 'AdGuardHome frontend'
# TODO(a.garipov): Use more fine-grained artifact rules.
'artifacts':
- 'name': 'AdGuardHome dists'
'pattern': 'dist/**'
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerGo}'
'volumes':
@ -93,9 +99,6 @@
set -e -f -u -x
# Explicitly checkout the revision that we need.
git checkout "${bamboo.repository.revision.number}"
# Run the build with the specified channel.
echo "${bamboo.gpgSecretKeyPart1}${bamboo.gpgSecretKeyPart2}"\
| awk '{ gsub(/\\n/, "\n"); print; }'\
@ -108,12 +111,6 @@
PARALLELISM=1\
VERBOSE=2\
build-release
# TODO(a.garipov): Use more fine-grained artifact rules.
'artifacts':
- 'name': 'AdGuardHome dists'
'pattern': 'dist/**'
'shared': true
'required': true
'requirements':
- 'adg-docker': 'true'
@ -132,13 +129,6 @@
set -e -f -u -x
COMMIT="${bamboo.repository.revision.number}"
export COMMIT
readonly COMMIT
# Explicitly checkout the revision that we need.
git checkout "$COMMIT"
# Install Qemu, create builder.
docker version -f '{{ .Server.Experimental }}'
docker buildx rm buildx-builder || :
@ -159,6 +149,7 @@
# Prepare and push the build.
env\
CHANNEL="${bamboo.channel}"\
COMMIT="${bamboo.repository.revision.number}"\
DIST_DIR='dist'\
DOCKER_IMAGE_NAME='adguard/adguardhome'\
DOCKER_OUTPUT="type=image,name=adguard/adguardhome,push=true"\
@ -274,7 +265,8 @@
# need to build a few of these.
'variables':
'channel': 'beta'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerFrontend': 'adguard/home-js-builder:1.1'
'dockerGo': 'adguard/go-builder:1.21.8--1'
# release-vX.Y.Z branches are the branches from which the actual final
# release is built.
- '^release-v[0-9]+\.[0-9]+\.[0-9]+':
@ -289,4 +281,5 @@
# are the ones that actually get released.
'variables':
'channel': 'release'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerFrontend': 'adguard/home-js-builder:1.1'
'dockerGo': 'adguard/go-builder:1.21.8--1'

View file

@ -10,7 +10,7 @@
# Make sure to sync any changes with the branch overrides below.
'variables':
'channel': 'edge'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerSnap': 'adguard/snap-builder:1.1'
'snapcraftChannel': 'edge'
'stages':
@ -53,7 +53,7 @@
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerGo}'
'image': '${bamboo.dockerSnap}'
'key': 'DR'
'other':
'clean-working-dir': true
@ -99,7 +99,7 @@
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerGo}'
'image': '${bamboo.dockerSnap}'
'key': 'BP'
'other':
'clean-working-dir': true
@ -127,7 +127,7 @@
- 'artifact': 'armhf_snap'
- 'artifact': 'arm64_snap'
'docker':
'image': '${bamboo.dockerGo}'
'image': '${bamboo.dockerSnap}'
'key': 'PTS'
'other':
'clean-working-dir': true
@ -191,7 +191,7 @@
# need to build a few of these.
'variables':
'channel': 'beta'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerSnap': 'adguard/snap-builder:1.1'
'snapcraftChannel': 'beta'
# release-vX.Y.Z branches are the branches from which the actual final
# release is built.
@ -207,5 +207,5 @@
# are the ones that actually get released.
'variables':
'channel': 'release'
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerSnap': 'adguard/snap-builder:1.1'
'snapcraftChannel': 'candidate'

View file

@ -5,7 +5,8 @@
'key': 'AHBRTSPECS'
'name': 'AdGuard Home - Build and run tests'
'variables':
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerFrontend': 'adguard/home-js-builder:1.1'
'dockerGo': 'adguard/go-builder:1.21.8--1'
'channel': 'development'
'stages':
@ -13,7 +14,14 @@
'manual': false
'final': false
'jobs':
- 'Test'
- 'Test frontend'
- 'Test backend'
- 'Frontend':
manual: false
final: false
jobs:
- 'Build frontend'
- 'Artifact':
manual: false
@ -21,14 +29,12 @@
jobs:
- 'Artifact'
'Test':
'Test frontend':
'docker':
'image': '${bamboo.dockerGo}'
'image': '${bamboo.dockerFrontend}'
'volumes':
'${system.YARN_DIR}': '${bamboo.cacheYarn}'
'${system.GO_CACHE_DIR}': '${bamboo.cacheGo}'
'${system.GO_PKG_CACHE_DIR}': '${bamboo.cacheGoPkg}'
'key': 'TEST'
'key': 'JSTEST'
'other':
'clean-working-dir': true
'tasks':
@ -42,13 +48,91 @@
set -e -f -u -x
make VERBOSE=1 ci go-tools lint
make VERBOSE=1 js-deps js-lint js-test
'final-tasks':
- 'clean'
'requirements':
- 'adg-docker': 'true'
'Test backend':
'docker':
'image': '${bamboo.dockerGo}'
'volumes':
'${system.GO_CACHE_DIR}': '${bamboo.cacheGo}'
'${system.GO_PKG_CACHE_DIR}': '${bamboo.cacheGoPkg}'
'key': 'GOTEST'
'other':
'clean-working-dir': true
'tasks':
- 'checkout':
'force-clean-build': true
- 'script':
'interpreter': 'SHELL'
'scripts':
- |
#!/bin/sh
set -e -f -u -x
make\
GOMAXPROCS=1\
VERBOSE=1\
go-deps go-tools go-lint
make\
VERBOSE=1\
go-test
'final-tasks':
- 'clean'
'requirements':
- 'adg-docker': 'true'
'Build frontend':
'artifacts':
- 'name': 'AdGuardHome frontend'
'pattern': 'build/**'
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerFrontend}'
'volumes':
'${system.YARN_DIR}': '${bamboo.cacheYarn}'
'key': 'BF'
'other':
'clean-working-dir': true
'tasks':
- 'checkout':
'force-clean-build': true
- 'script':
'interpreter': 'SHELL'
'scripts':
- |-
#!/bin/sh
set -e -f -u -x
make\
VERBOSE=1\
js-deps js-build
'requirements':
- 'adg-docker': 'true'
'Artifact':
'artifact-subscriptions':
- 'artifact': 'AdGuardHome frontend'
'artifacts':
- 'name': 'AdGuardHome_windows_amd64'
'pattern': 'dist/AdGuardHome_windows_amd64.zip'
'shared': true
'required': true
- 'name': 'AdGuardHome_darwin_amd64'
'pattern': 'dist/AdGuardHome_darwin_amd64.zip'
'shared': true
'required': true
- 'name': 'AdGuardHome_linux_amd64'
'pattern': 'dist/AdGuardHome_linux_amd64.tar.gz'
'shared': true
'required': true
'docker':
'image': '${bamboo.dockerGo}'
'volumes':
@ -70,25 +154,13 @@
make\
ARCH="amd64"\
OS="windows darwin linux"\
CHANNEL=${bamboo.channel}\
SIGN=0\
FRONTEND_PREBUILT=1\
OS="windows darwin linux"\
PARALLELISM=1\
SIGN=0\
VERBOSE=2\
build-release
'artifacts':
- 'name': 'AdGuardHome_windows_amd64'
'pattern': 'dist/AdGuardHome_windows_amd64.zip'
'shared': true
'required': true
- 'name': 'AdGuardHome_darwin_amd64'
'pattern': 'dist/AdGuardHome_darwin_amd64.zip'
'shared': true
'required': true
- 'name': 'AdGuardHome_linux_amd64'
'pattern': 'dist/AdGuardHome_linux_amd64.tar.gz'
'shared': true
'required': true
'requirements':
- 'adg-docker': 'true'
@ -122,5 +194,6 @@
# Set the default release channel on the release branch to beta, as we
# may need to build a few of these.
'variables':
'dockerGo': 'adguard/golang-ubuntu:8.1'
'dockerFrontend': 'adguard/home-js-builder:1.1'
'dockerGo': 'adguard/go-builder:1.21.8--1'
'channel': 'candidate'

4
go.mod
View file

@ -3,8 +3,8 @@ module github.com/AdguardTeam/AdGuardHome
go 1.21.8
require (
github.com/AdguardTeam/dnsproxy v0.66.0
github.com/AdguardTeam/golibs v0.20.2
github.com/AdguardTeam/dnsproxy v0.67.0
github.com/AdguardTeam/golibs v0.21.0
github.com/AdguardTeam/urlfilter v0.18.0
github.com/NYTimes/gziphandler v1.1.1
github.com/ameshkov/dnscrypt/v2 v2.2.7

8
go.sum
View file

@ -1,7 +1,7 @@
github.com/AdguardTeam/dnsproxy v0.66.0 h1:RyUbyDxRSXBFjVG1l2/4HV3I98DtfIgpnZkgXkgHKnc=
github.com/AdguardTeam/dnsproxy v0.66.0/go.mod h1:ZThEXbMUlP1RxfwtNW30ItPAHE6OF4YFygK8qjU/cvY=
github.com/AdguardTeam/golibs v0.20.2 h1:9gThBFyuELf2ohRnUNeQGQsVBYI7YslaRLUFwVaUj8E=
github.com/AdguardTeam/golibs v0.20.2/go.mod h1:/votX6WK1PdcZ3T2kBOPjPCGmfhlKixhI6ljYrFRPvI=
github.com/AdguardTeam/dnsproxy v0.67.0 h1:7oKfcA8sm9d1N4qvhsNmQWBX4+fs3sX4cAnERmBXEbw=
github.com/AdguardTeam/dnsproxy v0.67.0/go.mod h1:XLfD6IpSplUZZ+f5vhWSJW1mp4wm+KkHWiMo9w7U1Ls=
github.com/AdguardTeam/golibs v0.21.0 h1:0swWyNaHTmT7aMwffKd9d54g4wBd8Oaj0fl+5l/PRdE=
github.com/AdguardTeam/golibs v0.21.0/go.mod h1:/votX6WK1PdcZ3T2kBOPjPCGmfhlKixhI6ljYrFRPvI=
github.com/AdguardTeam/urlfilter v0.18.0 h1:ZZzwODC/ADpjJSODxySrrUnt/fvOCfGFaCW6j+wsGfQ=
github.com/AdguardTeam/urlfilter v0.18.0/go.mod h1:IXxBwedLiZA2viyHkaFxY/8mjub0li2PXRg8a3d9Z1s=
github.com/NYTimes/gziphandler v1.1.1 h1:ZUDjpQae29j0ryrS0u/B8HZfJBtBQHjqw2rQ2cqUQ3I=

View file

@ -1,10 +1,7 @@
package aghnet
import (
"fmt"
"strings"
"github.com/AdguardTeam/golibs/stringutil"
)
// NormalizeDomain returns a lowercased version of host without the final dot,
@ -19,25 +16,3 @@ func NormalizeDomain(host string) (norm string) {
return strings.ToLower(strings.TrimSuffix(host, "."))
}
// NewDomainNameSet returns nil and error, if list has duplicate or empty domain
// name. Otherwise returns a set, which contains domain names normalized using
// [NormalizeDomain].
func NewDomainNameSet(list []string) (set *stringutil.Set, err error) {
set = stringutil.NewSet()
for i, host := range list {
if host == "" {
return nil, fmt.Errorf("at index %d: hostname is empty", i)
}
host = NormalizeDomain(host)
if set.Has(host) {
return nil, fmt.Errorf("duplicate hostname %q at index %d", host, i)
}
set.Add(host)
}
return set, nil
}

View file

@ -1,59 +0,0 @@
package aghnet_test
import (
"testing"
"github.com/AdguardTeam/AdGuardHome/internal/aghnet"
"github.com/AdguardTeam/golibs/testutil"
"github.com/stretchr/testify/assert"
)
func TestNewDomainNameSet(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
wantErrMsg string
in []string
}{{
name: "nil",
wantErrMsg: "",
in: nil,
}, {
name: "success",
wantErrMsg: "",
in: []string{
"Domain.Example",
".",
},
}, {
name: "dups",
wantErrMsg: `duplicate hostname "domain.example" at index 1`,
in: []string{
"Domain.Example",
"domain.example",
},
}, {
name: "bad_domain",
wantErrMsg: "at index 0: hostname is empty",
in: []string{
"",
},
}}
for _, tc := range testCases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
set, err := aghnet.NewDomainNameSet(tc.in)
testutil.AssertErrorMsg(t, tc.wantErrMsg, err)
if err != nil {
return
}
for _, host := range tc.in {
assert.Truef(t, set.Has(aghnet.NormalizeDomain(host)), "%q not matched", host)
}
})
}
}

View file

@ -5,8 +5,8 @@ import (
"io"
"io/fs"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/stringutil"
)
// FileWalker is the signature of a function called for files in the file tree.
@ -56,7 +56,7 @@ func checkFile(
// srcSet. srcSet must be non-nil.
func handlePatterns(
fsys fs.FS,
srcSet *stringutil.Set,
srcSet *container.MapSet[string],
patterns ...string,
) (sub []string, err error) {
sub = make([]string, 0, len(patterns))
@ -87,7 +87,7 @@ func handlePatterns(
func (fw FileWalker) Walk(fsys fs.FS, initial ...string) (ok bool, err error) {
// The slice of sources keeps the order in which the files are walked since
// srcSet.Values() returns strings in undefined order.
srcSet := stringutil.NewSet()
srcSet := container.NewMapSet[string]()
var src []string
src, err = handlePatterns(fsys, srcSet, initial...)
if err != nil {

View file

@ -6,10 +6,10 @@ import (
"io/fs"
"path/filepath"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/osutil"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/fsnotify/fsnotify"
)
@ -46,7 +46,7 @@ type osWatcher struct {
events chan event
// files is the set of tracked files.
files *stringutil.Set
files *container.MapSet[string]
}
// osWatcherPref is a prefix for logging and wrapping errors in osWathcer's
@ -67,7 +67,7 @@ func NewOSWritesWatcher() (w FSWatcher, err error) {
return &osWatcher{
watcher: watcher,
events: make(chan event, 1),
files: stringutil.NewSet(),
files: container.NewMapSet[string](),
}, nil
}

View file

@ -12,10 +12,10 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/safesearch"
"github.com/AdguardTeam/dnsproxy/proxy"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/netutil"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/google/uuid"
)
@ -98,7 +98,7 @@ type Persistent struct {
}
// SetTags sets the tags if they are known, otherwise logs an unknown tag.
func (c *Persistent) SetTags(tags []string, known *stringutil.Set) {
func (c *Persistent) SetTags(tags []string, known *container.MapSet[string]) {
for _, t := range tags {
if !known.Has(t) {
log.Info("skipping unknown tag %q", t)

View file

@ -5,10 +5,12 @@ import (
"fmt"
"net/http"
"net/netip"
"slices"
"strings"
"github.com/AdguardTeam/AdGuardHome/internal/aghalg"
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/urlfilter"
@ -16,22 +18,19 @@ import (
"github.com/AdguardTeam/urlfilter/rules"
)
// unit is a convenient alias for struct{}
type unit = struct{}
// accessManager controls IP and client blocking that takes place before all
// other processing. An accessManager is safe for concurrent use.
type accessManager struct {
allowedIPs map[netip.Addr]unit
blockedIPs map[netip.Addr]unit
allowedIPs *container.MapSet[netip.Addr]
blockedIPs *container.MapSet[netip.Addr]
allowedClientIDs *stringutil.Set
blockedClientIDs *stringutil.Set
allowedClientIDs *container.MapSet[string]
blockedClientIDs *container.MapSet[string]
// TODO(s.chzhen): Use [aghnet.IgnoreEngine].
blockedHostsEng *urlfilter.DNSEngine
// TODO(a.garipov): Create a type for a set of IP networks.
// TODO(a.garipov): Create a type for an efficient tree set of IP networks.
allowedNets []netip.Prefix
blockedNets []netip.Prefix
}
@ -40,15 +39,15 @@ type accessManager struct {
// which may be an IP address, a CIDR, or a ClientID.
func processAccessClients(
clientStrs []string,
ips map[netip.Addr]unit,
ips *container.MapSet[netip.Addr],
nets *[]netip.Prefix,
clientIDs *stringutil.Set,
clientIDs *container.MapSet[string],
) (err error) {
for i, s := range clientStrs {
var ip netip.Addr
var ipnet netip.Prefix
if ip, err = netip.ParseAddr(s); err == nil {
ips[ip] = unit{}
ips.Add(ip)
} else if ipnet, err = netip.ParsePrefix(s); err == nil {
*nets = append(*nets, ipnet)
} else {
@ -67,11 +66,11 @@ func processAccessClients(
// newAccessCtx creates a new accessCtx.
func newAccessCtx(allowed, blocked, blockedHosts []string) (a *accessManager, err error) {
a = &accessManager{
allowedIPs: map[netip.Addr]unit{},
blockedIPs: map[netip.Addr]unit{},
allowedIPs: container.NewMapSet[netip.Addr](),
blockedIPs: container.NewMapSet[netip.Addr](),
allowedClientIDs: stringutil.NewSet(),
blockedClientIDs: stringutil.NewSet(),
allowedClientIDs: container.NewMapSet[string](),
blockedClientIDs: container.NewMapSet[string](),
}
err = processAccessClients(allowed, a.allowedIPs, &a.allowedNets, a.allowedClientIDs)
@ -109,7 +108,7 @@ func newAccessCtx(allowed, blocked, blockedHosts []string) (a *accessManager, er
// allowlistMode returns true if this *accessCtx is in the allowlist mode.
func (a *accessManager) allowlistMode() (ok bool) {
return len(a.allowedIPs) != 0 || a.allowedClientIDs.Len() != 0 || len(a.allowedNets) != 0
return a.allowedIPs.Len() != 0 || a.allowedClientIDs.Len() != 0 || len(a.allowedNets) != 0
}
// isBlockedClientID returns true if the ClientID should be blocked.
@ -152,7 +151,7 @@ func (a *accessManager) isBlockedIP(ip netip.Addr) (blocked bool, rule string) {
ipnets = a.allowedNets
}
if _, ok := ips[ip]; ok {
if ips.Has(ip) {
return blocked, ip.String()
}
@ -176,9 +175,9 @@ func (s *Server) accessListJSON() (j accessListJSON) {
defer s.serverLock.RUnlock()
return accessListJSON{
AllowedClients: stringutil.CloneSlice(s.conf.AllowedClients),
DisallowedClients: stringutil.CloneSlice(s.conf.DisallowedClients),
BlockedHosts: stringutil.CloneSlice(s.conf.BlockedHosts),
AllowedClients: slices.Clone(s.conf.AllowedClients),
DisallowedClients: slices.Clone(s.conf.DisallowedClients),
BlockedHosts: slices.Clone(s.conf.BlockedHosts),
}
}

View file

@ -19,6 +19,7 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/dnsproxy/proxy"
"github.com/AdguardTeam/dnsproxy/upstream"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/netutil"
@ -461,26 +462,27 @@ func (s *Server) prepareIpsetListSettings() (err error) {
// unspecPorts if its address is unspecified.
func collectListenAddr(
addrPort netip.AddrPort,
addrs map[netip.AddrPort]unit,
unspecPorts map[uint16]unit,
addrs *container.MapSet[netip.AddrPort],
unspecPorts *container.MapSet[uint16],
) {
if addrPort == (netip.AddrPort{}) {
return
}
addrs[addrPort] = unit{}
addrs.Add(addrPort)
if addrPort.Addr().IsUnspecified() {
unspecPorts[addrPort.Port()] = unit{}
unspecPorts.Add(addrPort.Port())
}
}
// collectDNSAddrs returns configured set of listening addresses. It also
// returns a set of ports of each unspecified listening address.
func (conf *ServerConfig) collectDNSAddrs() (addrs mapAddrPortSet, unspecPorts map[uint16]unit) {
// TODO(e.burkov): Perhaps, we shouldn't allocate as much memory, since the
// TCP and UDP listening addresses are currently the same.
addrs = make(map[netip.AddrPort]unit, len(conf.TCPListenAddrs)+len(conf.UDPListenAddrs))
unspecPorts = map[uint16]unit{}
func (conf *ServerConfig) collectDNSAddrs() (
addrs *container.MapSet[netip.AddrPort],
unspecPorts *container.MapSet[uint16],
) {
addrs = container.NewMapSet[netip.AddrPort]()
unspecPorts = container.NewMapSet[uint16]()
for _, laddr := range conf.TCPListenAddrs {
collectListenAddr(laddr.AddrPort(), addrs, unspecPorts)
@ -511,26 +513,12 @@ type emptyAddrPortSet struct{}
// Has implements the [addrPortSet] interface for [emptyAddrPortSet].
func (emptyAddrPortSet) Has(_ netip.AddrPort) (ok bool) { return false }
// mapAddrPortSet is the [addrPortSet] containing values of [netip.AddrPort] as
// keys of a map.
type mapAddrPortSet map[netip.AddrPort]unit
// type check
var _ addrPortSet = mapAddrPortSet{}
// Has implements the [addrPortSet] interface for [mapAddrPortSet].
func (m mapAddrPortSet) Has(addrPort netip.AddrPort) (ok bool) {
_, ok = m[addrPort]
return ok
}
// combinedAddrPortSet is the [addrPortSet] defined by some IP addresses along
// with ports, any combination of which is considered being in the set.
type combinedAddrPortSet struct {
// TODO(e.burkov): Use sorted slices in combination with binary search.
ports map[uint16]unit
addrs []netip.Addr
// TODO(e.burkov): Use container.SliceSet when available.
ports *container.MapSet[uint16]
addrs *container.MapSet[netip.Addr]
}
// type check
@ -538,9 +526,7 @@ var _ addrPortSet = (*combinedAddrPortSet)(nil)
// Has implements the [addrPortSet] interface for [*combinedAddrPortSet].
func (m *combinedAddrPortSet) Has(addrPort netip.AddrPort) (ok bool) {
_, ok = m.ports[addrPort.Port()]
return ok && slices.Contains(m.addrs, addrPort.Addr())
return m.ports.Has(addrPort.Port()) && m.addrs.Has(addrPort.Addr())
}
// filterOut filters out all the upstreams that match um. It returns all the
@ -578,11 +564,11 @@ func filterOutAddrs(upsConf *proxy.UpstreamConfig, set addrPortSet) (err error)
func (conf *ServerConfig) ourAddrsSet() (m addrPortSet, err error) {
addrs, unspecPorts := conf.collectDNSAddrs()
switch {
case len(addrs) == 0:
case addrs.Len() == 0:
log.Debug("dnsforward: no listen addresses")
return emptyAddrPortSet{}, nil
case len(unspecPorts) == 0:
case unspecPorts.Len() == 0:
log.Debug("dnsforward: filtering out addresses %s", addrs)
return addrs, nil
@ -598,7 +584,7 @@ func (conf *ServerConfig) ourAddrsSet() (m addrPortSet, err error) {
return &combinedAddrPortSet{
ports: unspecPorts,
addrs: ifaceAddrs,
addrs: container.NewMapSet(ifaceAddrs...),
}, nil
}
}

View file

@ -308,13 +308,13 @@ func (s *Server) WriteDiskConfig(c *Config) {
sc := s.conf.Config
*c = sc
c.RatelimitWhitelist = slices.Clone(sc.RatelimitWhitelist)
c.BootstrapDNS = stringutil.CloneSlice(sc.BootstrapDNS)
c.FallbackDNS = stringutil.CloneSlice(sc.FallbackDNS)
c.AllowedClients = stringutil.CloneSlice(sc.AllowedClients)
c.DisallowedClients = stringutil.CloneSlice(sc.DisallowedClients)
c.BlockedHosts = stringutil.CloneSlice(sc.BlockedHosts)
c.BootstrapDNS = slices.Clone(sc.BootstrapDNS)
c.FallbackDNS = slices.Clone(sc.FallbackDNS)
c.AllowedClients = slices.Clone(sc.AllowedClients)
c.DisallowedClients = slices.Clone(sc.DisallowedClients)
c.BlockedHosts = slices.Clone(sc.BlockedHosts)
c.TrustedProxies = slices.Clone(sc.TrustedProxies)
c.UpstreamDNS = stringutil.CloneSlice(sc.UpstreamDNS)
c.UpstreamDNS = slices.Clone(sc.UpstreamDNS)
}
// LocalPTRResolvers returns the current local PTR resolver configuration.
@ -322,7 +322,7 @@ func (s *Server) LocalPTRResolvers() (localPTRResolvers []string) {
s.serverLock.RLock()
defer s.serverLock.RUnlock()
return stringutil.CloneSlice(s.conf.LocalPTRResolvers)
return slices.Clone(s.conf.LocalPTRResolvers)
}
// AddrProcConfig returns the current address processing configuration. Only

View file

@ -474,8 +474,6 @@ func (s *Server) setConfig(dc *jsonDNSConfig) (shouldRestart bool) {
if dc.UpstreamMode != nil {
s.conf.UpstreamMode = mustParseUpstreamMode(*dc.UpstreamMode)
} else {
s.conf.UpstreamMode = UpstreamModeLoadBalance
}
if dc.EDNSCSUseCustom != nil && *dc.EDNSCSUseCustom {

View file

@ -29,6 +29,10 @@ import (
"github.com/stretchr/testify/require"
)
// TODO(e.burkov): Use the better approach to testdata with a separate
// directory for each test, and a separate file for each subtest. See the
// [configmigrate] package.
// emptySysResolvers is an empty [SystemResolvers] implementation that always
// returns nil.
type emptySysResolvers struct{}

View file

@ -8,6 +8,7 @@ import (
"time"
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/AdGuardHome/internal/schedule"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter/rules"
@ -28,7 +29,7 @@ func initBlockedServices() {
for i, s := range blockedServices {
netRules := make([]*rules.NetworkRule, 0, len(s.Rules))
for _, text := range s.Rules {
rule, err := rules.NewNetworkRule(text, BlockedSvcsListID)
rule, err := rules.NewNetworkRule(text, rulelist.URLFilterIDBlockedService)
if err != nil {
log.Error("parsing blocked service %q rule %q: %s", s.ID, text, err)

View file

@ -30,7 +30,7 @@ func (d *DNSFilter) processDNSRewrites(dnsr []*rules.NetworkRule) (res Result) {
if dr.NewCNAME != "" {
// NewCNAME rules have a higher priority than other rules.
rules = []*ResultRule{{
FilterListID: int64(nr.GetFilterListID()),
FilterListID: nr.GetFilterListID(),
Text: nr.RuleText,
}}
@ -46,14 +46,14 @@ func (d *DNSFilter) processDNSRewrites(dnsr []*rules.NetworkRule) (res Result) {
dnsrr.RCode = dr.RCode
dnsrr.Response[dr.RRType] = append(dnsrr.Response[dr.RRType], dr.Value)
rules = append(rules, &ResultRule{
FilterListID: int64(nr.GetFilterListID()),
FilterListID: nr.GetFilterListID(),
Text: nr.RuleText,
})
default:
// RcodeRefused and other such codes have higher priority. Return
// immediately.
rules = []*ResultRule{{
FilterListID: int64(nr.GetFilterListID()),
FilterListID: nr.GetFilterListID(),
Text: nr.RuleText,
}}
dnsrr = &DNSRewriteResult{

View file

@ -13,20 +13,15 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghrenameio"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
)
// filterDir is the subdirectory of a data directory to store downloaded
// filters.
const filterDir = "filters"
// nextFilterID is a way to seed a unique ID generation.
//
// TODO(e.burkov): Use more deterministic approach.
var nextFilterID = time.Now().Unix()
// FilterYAML represents a filter list in the configuration file.
//
// TODO(e.burkov): Investigate if the field ordering is important.
@ -50,7 +45,10 @@ func (filter *FilterYAML) unload() {
// Path to the filter contents
func (filter *FilterYAML) Path(dataDir string) string {
return filepath.Join(dataDir, filterDir, strconv.FormatInt(filter.ID, 10)+".txt")
return filepath.Join(
dataDir,
filterDir,
strconv.FormatInt(int64(filter.ID), 10)+".txt")
}
// ensureName sets provided title or default name for the filter if it doesn't
@ -217,7 +215,10 @@ func (d *DNSFilter) loadFilters(array []FilterYAML) {
for i := range array {
filter := &array[i] // otherwise we're operating on a copy
if filter.ID == 0 {
filter.ID = assignUniqueFilterID()
newID := d.idGen.next()
log.Info("filtering: warning: filter at index %d has no id; assigning to %d", i, newID)
filter.ID = newID
}
if !filter.Enabled {
@ -233,7 +234,7 @@ func (d *DNSFilter) loadFilters(array []FilterYAML) {
}
func deduplicateFilters(filters []FilterYAML) (deduplicated []FilterYAML) {
urls := stringutil.NewSet()
urls := container.NewMapSet[string]()
lastIdx := 0
for _, filter := range filters {
@ -247,22 +248,6 @@ func deduplicateFilters(filters []FilterYAML) (deduplicated []FilterYAML) {
return filters[:lastIdx]
}
// Set the next filter ID to max(filter.ID) + 1
func updateUniqueFilterID(filters []FilterYAML) {
for _, filter := range filters {
if nextFilterID < filter.ID {
nextFilterID = filter.ID + 1
}
}
}
// TODO(e.burkov): Improve this inexhaustible source of races.
func assignUniqueFilterID() int64 {
value := nextFilterID
nextFilterID++
return value
}
// tryRefreshFilters is like [refreshFilters], but backs down if the update is
// already going on.
//
@ -608,7 +593,7 @@ func (d *DNSFilter) EnableFilters(async bool) {
func (d *DNSFilter) enableFiltersLocked(async bool) {
filters := make([]Filter, 1, len(d.conf.Filters)+len(d.conf.WhitelistFilters)+1)
filters[0] = Filter{
ID: CustomListID,
ID: rulelist.URLFilterIDCustom,
Data: []byte(strings.Join(d.conf.UserRules, "\n")),
}

View file

@ -20,11 +20,11 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/hostsfile"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/mathutil"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/golibs/syncutil"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
@ -32,19 +32,6 @@ import (
"github.com/miekg/dns"
)
// The IDs of built-in filter lists.
//
// Keep in sync with client/src/helpers/constants.js.
// TODO(d.kolyshev): Add RewritesListID and don't forget to keep in sync.
const (
CustomListID = -iota
SysHostsListID
BlockedSvcsListID
ParentalListID
SafeBrowsingListID
SafeSearchListID
)
// ServiceEntry - blocked service array element
type ServiceEntry struct {
Name string
@ -232,6 +219,9 @@ type Checker interface {
// DNSFilter matches hostnames and DNS requests against filtering rules.
type DNSFilter struct {
// idGen is used to generate IDs for package urlfilter.
idGen *idGenerator
// bufPool is a pool of buffers used for filtering-rule list parsing.
bufPool *syncutil.Pool[[]byte]
@ -278,7 +268,7 @@ type Filter struct {
Data []byte `yaml:"-"`
// ID is automatically assigned when filter is added using nextFilterID.
ID int64 `yaml:"id"`
ID rulelist.URLFilterID `yaml:"id"`
}
// Reason holds an enum detailing why it was filtered or not filtered
@ -530,11 +520,13 @@ func (d *DNSFilter) ParentalBlockHost() (host string) {
type ResultRule struct {
// Text is the text of the rule.
Text string `json:",omitempty"`
// IP is the host IP. It is nil unless the rule uses the
// /etc/hosts syntax or the reason is FilteredSafeSearch.
IP netip.Addr `json:",omitempty"`
// FilterListID is the ID of the rule's filter list.
FilterListID int64 `json:",omitempty"`
FilterListID rulelist.URLFilterID `json:",omitempty"`
}
// Result contains the result of a request check.
@ -637,7 +629,7 @@ func (d *DNSFilter) processRewrites(host string, qtype uint16) (res Result) {
res.Reason = Rewritten
cnames := stringutil.NewSet()
cnames := container.NewMapSet[string]()
origHost := host
for matched && len(rewrites) > 0 && rewrites[0].Type == dns.TypeCNAME {
rw := rewrites[0]
@ -705,7 +697,7 @@ func matchBlockedServicesRules(
ruleText := rule.Text()
res.Rules = []*ResultRule{{
FilterListID: int64(rule.GetFilterListID()),
FilterListID: rule.GetFilterListID(),
Text: ruleText,
}}
@ -970,7 +962,7 @@ func makeResult(matchedRules []rules.Rule, reason Reason) (res Result) {
resRules := make([]*ResultRule, len(matchedRules))
for i, mr := range matchedRules {
resRules[i] = &ResultRule{
FilterListID: int64(mr.GetFilterListID()),
FilterListID: mr.GetFilterListID(),
Text: mr.Text(),
}
}
@ -991,6 +983,7 @@ func InitModule() {
// be non-nil.
func New(c *Config, blockFilters []Filter) (d *DNSFilter, err error) {
d = &DNSFilter{
idGen: newIDGenerator(int32(time.Now().Unix())),
bufPool: syncutil.NewSlicePool[byte](rulelist.DefaultRuleBufSize),
refreshLock: &sync.Mutex{},
safeBrowsingChecker: c.SafeBrowsingChecker,
@ -1054,8 +1047,8 @@ func New(c *Config, blockFilters []Filter) (d *DNSFilter, err error) {
d.conf.Filters = deduplicateFilters(d.conf.Filters)
d.conf.WhitelistFilters = deduplicateFilters(d.conf.WhitelistFilters)
updateUniqueFilterID(d.conf.Filters)
updateUniqueFilterID(d.conf.WhitelistFilters)
d.idGen.fix(d.conf.Filters)
d.idGen.fix(d.conf.WhitelistFilters)
return d, nil
}
@ -1139,7 +1132,7 @@ func (d *DNSFilter) checkSafeBrowsing(
res = Result{
Rules: []*ResultRule{{
Text: "adguard-malware-shavar",
FilterListID: SafeBrowsingListID,
FilterListID: rulelist.URLFilterIDSafeBrowsing,
}},
Reason: FilteredSafeBrowsing,
IsFiltered: true,
@ -1171,7 +1164,7 @@ func (d *DNSFilter) checkParental(
res = Result{
Rules: []*ResultRule{{
Text: "parental CATEGORY_BLACKLISTED",
FilterListID: ParentalListID,
FilterListID: rulelist.URLFilterIDParentalControl,
}},
Reason: FilteredParental,
IsFiltered: true,

View file

@ -4,6 +4,7 @@ import (
"fmt"
"net/netip"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/hostsfile"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/netutil"
@ -66,7 +67,7 @@ func hostsRewrites(
vals = append(vals, name)
rls = append(rls, &ResultRule{
Text: fmt.Sprintf("%s %s", addr, name),
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
})
}
@ -84,7 +85,7 @@ func hostsRewrites(
}
rls = append(rls, &ResultRule{
Text: fmt.Sprintf("%s %s", addr, host),
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
})
}

View file

@ -8,6 +8,7 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghnet"
"github.com/AdguardTeam/AdGuardHome/internal/aghtest"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/testutil"
"github.com/AdguardTeam/urlfilter/rules"
"github.com/miekg/dns"
@ -71,7 +72,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeA,
wantRules: []*ResultRule{{
Text: "1.2.3.4 v4.host.example",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{addrv4},
}, {
@ -80,7 +81,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeAAAA,
wantRules: []*ResultRule{{
Text: "::1 v6.host.example",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{addrv6},
}, {
@ -89,7 +90,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeAAAA,
wantRules: []*ResultRule{{
Text: "::ffff:1.2.3.4 mapped.host.example",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{addrMapped},
}, {
@ -98,7 +99,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypePTR,
wantRules: []*ResultRule{{
Text: "1.2.3.4 v4.host.example",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{"v4.host.example"},
}, {
@ -107,7 +108,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypePTR,
wantRules: []*ResultRule{{
Text: "::ffff:1.2.3.4 mapped.host.example",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{"mapped.host.example"},
}, {
@ -134,7 +135,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeAAAA,
wantRules: []*ResultRule{{
Text: fmt.Sprintf("%s v4.host.example", addrv4),
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: nil,
}, {
@ -143,7 +144,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeA,
wantRules: []*ResultRule{{
Text: fmt.Sprintf("%s v6.host.example", addrv6),
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: nil,
}, {
@ -164,7 +165,7 @@ func TestDNSFilter_CheckHost_hostsContainer(t *testing.T) {
dtyp: dns.TypeA,
wantRules: []*ResultRule{{
Text: "4.3.2.1 v4.host.with-dup",
FilterListID: SysHostsListID,
FilterListID: rulelist.URLFilterIDEtcHosts,
}},
wantResps: []rules.RRValue{addrv4Dup},
}}

View file

@ -13,6 +13,7 @@ import (
"time"
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/miekg/dns"
@ -86,7 +87,7 @@ func (d *DNSFilter) handleFilteringAddURL(w http.ResponseWriter, r *http.Request
Name: fj.Name,
white: fj.Whitelist,
Filter: Filter{
ID: assignUniqueFilterID(),
ID: d.idGen.next(),
},
}
@ -307,12 +308,12 @@ func (d *DNSFilter) handleFilteringRefresh(w http.ResponseWriter, r *http.Reques
}
type filterJSON struct {
URL string `json:"url"`
Name string `json:"name"`
LastUpdated string `json:"last_updated,omitempty"`
ID int64 `json:"id"`
RulesCount uint32 `json:"rules_count"`
Enabled bool `json:"enabled"`
URL string `json:"url"`
Name string `json:"name"`
LastUpdated string `json:"last_updated,omitempty"`
ID rulelist.URLFilterID `json:"id"`
RulesCount uint32 `json:"rules_count"`
Enabled bool `json:"enabled"`
}
type filteringConfig struct {
@ -388,8 +389,8 @@ func (d *DNSFilter) handleFilteringConfig(w http.ResponseWriter, r *http.Request
}
type checkHostRespRule struct {
Text string `json:"text"`
FilterListID int64 `json:"filter_list_id"`
Text string `json:"text"`
FilterListID rulelist.URLFilterID `json:"filter_list_id"`
}
type checkHostResp struct {
@ -412,7 +413,7 @@ type checkHostResp struct {
// FilterID is the ID of the rule's filter list.
//
// Deprecated: Use Rules[*].FilterListID.
FilterID int64 `json:"filter_id"`
FilterID rulelist.URLFilterID `json:"filter_id"`
}
func (d *DNSFilter) handleCheckHost(w http.ResponseWriter, r *http.Request) {

View file

@ -0,0 +1,74 @@
package filtering
import (
"fmt"
"sync/atomic"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/log"
)
// idGenerator generates filtering-list IDs in a way broadly compatible with the
// legacy approach of AdGuard Home.
//
// TODO(a.garipov): Get rid of this once we switch completely to the new
// rule-list architecture.
type idGenerator struct {
current *atomic.Int32
}
// newIDGenerator returns a new ID generator initialized with the given seed
// value.
func newIDGenerator(seed int32) (g *idGenerator) {
g = &idGenerator{
current: &atomic.Int32{},
}
g.current.Store(seed)
return g
}
// next returns the next ID from the generator. It is safe for concurrent use.
func (g *idGenerator) next() (id rulelist.URLFilterID) {
id32 := g.current.Add(1)
if id32 < 0 {
panic(fmt.Errorf("invalid current id value %d", id32))
}
return rulelist.URLFilterID(id32)
}
// fix ensures that flts all have unique IDs.
func (g *idGenerator) fix(flts []FilterYAML) {
set := container.NewMapSet[rulelist.URLFilterID]()
for i, f := range flts {
id := f.ID
if id == 0 {
id = g.next()
flts[i].ID = id
}
if !set.Has(id) {
set.Add(id)
continue
}
newID := g.next()
for set.Has(newID) {
newID = g.next()
}
log.Info(
"filtering: warning: filter at index %d has duplicate id %d; reassigning to %d",
i,
id,
newID,
)
flts[i].ID = newID
set.Add(newID)
}
}

View file

@ -0,0 +1,88 @@
package filtering
import (
"testing"
"github.com/AdguardTeam/AdGuardHome/internal/aghalg"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/stretchr/testify/assert"
)
func TestIDGenerator_Fix(t *testing.T) {
t.Parallel()
testCases := []struct {
name string
in []FilterYAML
}{{
name: "nil",
in: nil,
}, {
name: "empty",
in: []FilterYAML{},
}, {
name: "one_zero",
in: []FilterYAML{{}},
}, {
name: "two_zeros",
in: []FilterYAML{{}, {}},
}, {
name: "many_good",
in: []FilterYAML{{
Filter: Filter{
ID: 1,
},
}, {
Filter: Filter{
ID: 2,
},
}, {
Filter: Filter{
ID: 3,
},
}},
}, {
name: "two_dups",
in: []FilterYAML{{
Filter: Filter{
ID: 1,
},
}, {
Filter: Filter{
ID: 3,
},
}, {
Filter: Filter{
ID: 1,
},
}, {
Filter: Filter{
ID: 2,
},
}},
}}
for _, tc := range testCases {
tc := tc
t.Run(tc.name, func(t *testing.T) {
g := newIDGenerator(1)
g.fix(tc.in)
assertUniqueIDs(t, tc.in)
})
}
}
// assertUniqueIDs is a test helper that asserts that the IDs of filters are
// unique.
func assertUniqueIDs(t testing.TB, flts []FilterYAML) {
t.Helper()
uc := aghalg.UniqChecker[rulelist.URLFilterID]{}
for _, f := range flts {
uc.Add(f.ID)
}
assert.NoError(t, uc.Validate())
}

View file

@ -7,8 +7,8 @@ import (
"strings"
"sync"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
"github.com/AdguardTeam/urlfilter/rules"
@ -85,7 +85,7 @@ func (s *DefaultStorage) MatchRequest(dReq *urlfilter.DNSRequest) (rws []*rules.
}
// TODO(a.garipov): Check cnames for cycles on initialization.
cnames := stringutil.NewSet()
cnames := container.NewMapSet[string]()
host := dReq.Hostname
for len(rrules) > 0 && rrules[0].DNSRewrite != nil && rrules[0].DNSRewrite.NewCNAME != "" {
rule := rrules[0]

View file

@ -0,0 +1,254 @@
package rulelist
import (
"context"
"fmt"
"net/http"
"sync"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
"github.com/c2h5oh/datasize"
)
// Engine is a single DNS filter based on one or more rule lists. This
// structure contains the filtering engine combining several rule lists.
//
// TODO(a.garipov): Merge with [TextEngine] in some way?
type Engine struct {
// mu protects engine and storage.
//
// TODO(a.garipov): See if anything else should be protected.
mu *sync.RWMutex
// engine is the filtering engine.
engine *urlfilter.DNSEngine
// storage is the filtering-rule storage. It is saved here to close it.
storage *filterlist.RuleStorage
// name is the human-readable name of the engine, like "allowed", "blocked",
// or "custom".
name string
// filters is the data about rule filters in this engine.
filters []*Filter
}
// EngineConfig is the configuration for rule-list filtering engines created by
// combining refreshable filters.
type EngineConfig struct {
// Name is the human-readable name of this engine, like "allowed",
// "blocked", or "custom".
Name string
// Filters is the data about rule lists in this engine. There must be no
// other references to the elements of this slice.
Filters []*Filter
}
// NewEngine returns a new rule-list filtering engine. The engine is not
// refreshed, so a refresh should be performed before use.
func NewEngine(c *EngineConfig) (e *Engine) {
return &Engine{
mu: &sync.RWMutex{},
name: c.Name,
filters: c.Filters,
}
}
// Close closes the underlying rule-list engine as well as the rule lists.
func (e *Engine) Close() (err error) {
e.mu.Lock()
defer e.mu.Unlock()
if e.storage == nil {
return nil
}
err = e.storage.Close()
if err != nil {
return fmt.Errorf("closing engine %q: %w", e.name, err)
}
return nil
}
// FilterRequest returns the result of filtering req using the DNS filtering
// engine.
func (e *Engine) FilterRequest(
req *urlfilter.DNSRequest,
) (res *urlfilter.DNSResult, hasMatched bool) {
return e.currentEngine().MatchRequest(req)
}
// currentEngine returns the current filtering engine.
func (e *Engine) currentEngine() (enging *urlfilter.DNSEngine) {
e.mu.RLock()
defer e.mu.RUnlock()
return e.engine
}
// Refresh updates all rule lists in e. ctx is used for cancellation.
// parseBuf, cli, cacheDir, and maxSize are used for updates of rule-list
// filters; see [Filter.Refresh].
//
// TODO(a.garipov): Unexport and test in an internal test or through enigne
// tests.
func (e *Engine) Refresh(
ctx context.Context,
parseBuf []byte,
cli *http.Client,
cacheDir string,
maxSize datasize.ByteSize,
) (err error) {
defer func() { err = errors.Annotate(err, "updating engine %q: %w", e.name) }()
var filtersToRefresh []*Filter
for _, f := range e.filters {
if f.enabled {
filtersToRefresh = append(filtersToRefresh, f)
}
}
if len(filtersToRefresh) == 0 {
log.Info("filtering: updating engine %q: no rule-list filters", e.name)
return nil
}
engRefr := &engineRefresh{
httpCli: cli,
cacheDir: cacheDir,
engineName: e.name,
parseBuf: parseBuf,
maxSize: maxSize,
}
ruleLists, errs := engRefr.process(ctx, e.filters)
if isOneTimeoutError(errs) {
// Don't wrap the error since it's informative enough as is.
return err
}
storage, err := filterlist.NewRuleStorage(ruleLists)
if err != nil {
errs = append(errs, fmt.Errorf("creating rule storage: %w", err))
return errors.Join(errs...)
}
e.resetStorage(storage)
return errors.Join(errs...)
}
// resetStorage sets e.storage and e.engine and closes the previous storage.
// Errors from closing the previous storage are logged.
func (e *Engine) resetStorage(storage *filterlist.RuleStorage) {
e.mu.Lock()
defer e.mu.Unlock()
prevStorage := e.storage
e.storage, e.engine = storage, urlfilter.NewDNSEngine(storage)
if prevStorage == nil {
return
}
err := prevStorage.Close()
if err != nil {
log.Error("filtering: engine %q: closing old storage: %s", e.name, err)
}
}
// isOneTimeoutError returns true if the sole error in errs is either
// [context.Canceled] or [context.DeadlineExceeded].
func isOneTimeoutError(errs []error) (ok bool) {
if len(errs) != 1 {
return false
}
err := errs[0]
return errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded)
}
// engineRefresh represents a single ongoing engine refresh.
type engineRefresh struct {
httpCli *http.Client
cacheDir string
engineName string
parseBuf []byte
maxSize datasize.ByteSize
}
// process runs updates of all given rule-list filters. All errors are logged
// as they appear, since the update can take a significant amount of time.
// errs contains all errors that happened during the update, unless the context
// is canceled or its deadline is reached, in which case errs will only contain
// a single timeout error.
//
// TODO(a.garipov): Think of a better way to communicate the timeout condition?
func (r *engineRefresh) process(
ctx context.Context,
filters []*Filter,
) (ruleLists []filterlist.RuleList, errs []error) {
ruleLists = make([]filterlist.RuleList, 0, len(filters))
for i, f := range filters {
select {
case <-ctx.Done():
return nil, []error{fmt.Errorf("timeout after updating %d filters: %w", i, ctx.Err())}
default:
// Go on.
}
err := r.processFilter(ctx, f)
if err == nil {
ruleLists = append(ruleLists, f.ruleList)
continue
}
errs = append(errs, err)
// Also log immediately, since the update can take a lot of time.
log.Error(
"filtering: updating engine %q: rule list %s from url %q: %s\n",
r.engineName,
f.uid,
f.url,
err,
)
}
return ruleLists, errs
}
// processFilter runs an update of a single rule-list filter.
func (r *engineRefresh) processFilter(ctx context.Context, f *Filter) (err error) {
prevChecksum := f.checksum
parseRes, err := f.Refresh(ctx, r.parseBuf, r.httpCli, r.cacheDir, r.maxSize)
if err != nil {
return fmt.Errorf("updating %s: %w", f.uid, err)
}
if prevChecksum == parseRes.Checksum {
log.Info("filtering: engine %q: filter %q: no change", r.engineName, f.uid)
return nil
}
log.Info(
"filtering: updated engine %q: filter %q: %d bytes, %d rules",
r.engineName,
f.uid,
parseRes.BytesWritten,
parseRes.RulesCount,
)
return nil
}

View file

@ -0,0 +1,63 @@
package rulelist_test
import (
"context"
"net/http"
"testing"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/testutil"
"github.com/AdguardTeam/urlfilter"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestEngine_Refresh(t *testing.T) {
cacheDir := t.TempDir()
fileURL, srvURL := newFilterLocations(t, cacheDir, testRuleTextBlocked, testRuleTextBlocked2)
fileFlt := newFilter(t, fileURL, "File Filter")
httpFlt := newFilter(t, srvURL, "HTTP Filter")
eng := rulelist.NewEngine(&rulelist.EngineConfig{
Name: "Engine",
Filters: []*rulelist.Filter{fileFlt, httpFlt},
})
require.NotNil(t, eng)
testutil.CleanupAndRequireSuccess(t, eng.Close)
ctx, cancel := context.WithTimeout(context.Background(), testTimeout)
t.Cleanup(cancel)
buf := make([]byte, rulelist.DefaultRuleBufSize)
cli := &http.Client{
Timeout: testTimeout,
}
err := eng.Refresh(ctx, buf, cli, cacheDir, rulelist.DefaultMaxRuleListSize)
require.NoError(t, err)
fltReq := &urlfilter.DNSRequest{
Hostname: "blocked.example",
Answer: false,
DNSType: dns.TypeA,
}
fltRes, hasMatched := eng.FilterRequest(fltReq)
assert.True(t, hasMatched)
require.NotNil(t, fltRes)
fltReq = &urlfilter.DNSRequest{
Hostname: "blocked-2.example",
Answer: false,
DNSType: dns.TypeA,
}
fltRes, hasMatched = eng.FilterRequest(fltReq)
assert.True(t, hasMatched)
require.NotNil(t, fltRes)
}

View file

@ -14,7 +14,6 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghrenameio"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/ioutil"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter/filterlist"
"github.com/c2h5oh/datasize"
)
@ -52,8 +51,6 @@ type Filter struct {
checksum uint32
// enabled, if true, means that this rule-list filter is used for filtering.
//
// TODO(a.garipov): Take into account.
enabled bool
}
@ -106,6 +103,11 @@ func NewFilter(c *FilterConfig) (f *Filter, err error) {
// Refresh updates the data in the rule-list filter. parseBuf is the initial
// buffer used to parse information from the data. cli and maxSize are only
// used when f is a URL-based list.
//
// TODO(a.garipov): Unexport and test in an internal test or through enigne
// tests.
//
// TODO(a.garipov): Consider not returning parseRes.
func (f *Filter) Refresh(
ctx context.Context,
parseBuf []byte,
@ -300,39 +302,3 @@ func (f *Filter) Close() (err error) {
return f.ruleList.Close()
}
// filterUpdate represents a single ongoing rule-list filter update.
//
//lint:ignore U1000 TODO(a.garipov): Use.
type filterUpdate struct {
httpCli *http.Client
cacheDir string
name string
parseBuf []byte
maxSize datasize.ByteSize
}
// process runs an update of a single rule-list.
func (u *filterUpdate) process(ctx context.Context, f *Filter) (err error) {
prevChecksum := f.checksum
parseRes, err := f.Refresh(ctx, u.parseBuf, u.httpCli, u.cacheDir, u.maxSize)
if err != nil {
return fmt.Errorf("updating %s: %w", f.uid, err)
}
if prevChecksum == parseRes.Checksum {
log.Info("filtering: filter %q: filter %q: no change", u.name, f.uid)
return nil
}
log.Info(
"filtering: updated filter %q: filter %q: %d bytes, %d rules",
u.name,
f.uid,
parseRes.BytesWritten,
parseRes.RulesCount,
)
return nil
}

View file

@ -2,9 +2,7 @@ package rulelist_test
import (
"context"
"io"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
@ -20,23 +18,8 @@ func TestFilter_Refresh(t *testing.T) {
cacheDir := t.TempDir()
uid := rulelist.MustNewUID()
initialFile := filepath.Join(cacheDir, "initial.txt")
initialData := []byte(
testRuleTextTitle +
testRuleTextBlocked,
)
writeErr := os.WriteFile(initialFile, initialData, 0o644)
require.NoError(t, writeErr)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
pt := testutil.PanicT{}
_, err := io.WriteString(w, testRuleTextTitle+testRuleTextBlocked)
require.NoError(pt, err)
}))
srvURL, urlErr := url.Parse(srv.URL)
require.NoError(t, urlErr)
const fltData = testRuleTextTitle + testRuleTextBlocked
fileURL, srvURL := newFilterLocations(t, cacheDir, fltData, fltData)
testCases := []struct {
url *url.URL
@ -56,7 +39,7 @@ func TestFilter_Refresh(t *testing.T) {
name: "file",
url: &url.URL{
Scheme: "file",
Path: initialFile,
Path: fileURL.Path,
},
wantNewErrMsg: "",
}, {

View file

@ -25,6 +25,24 @@ const DefaultMaxRuleListSize = 64 * datasize.MB
// urlfilter.
type URLFilterID = int
// The IDs of built-in filter lists.
//
// NOTE: Do not change without the need for it and keep in sync with
// client/src/helpers/constants.js.
//
// TODO(a.garipov): Add type [URLFilterID] once it is used consistently in
// package filtering.
//
// TODO(d.kolyshev): Add URLFilterIDLegacyRewrite here and to the UI.
const (
URLFilterIDCustom URLFilterID = 0
URLFilterIDEtcHosts URLFilterID = -1
URLFilterIDBlockedService URLFilterID = -2
URLFilterIDParentalControl URLFilterID = -3
URLFilterIDSafeBrowsing URLFilterID = -4
URLFilterIDSafeSearch URLFilterID = -5
)
// UID is the type for the unique IDs of filtering-rule lists.
type UID uuid.UUID

View file

@ -1,11 +1,19 @@
package rulelist_test
import (
"io"
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"sync/atomic"
"testing"
"time"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/testutil"
"github.com/stretchr/testify/require"
)
func TestMain(m *testing.M) {
@ -35,3 +43,70 @@ const (
// See https://github.com/AdguardTeam/AdGuardHome/issues/6003.
testRuleTextCosmetic = "||cosmetic.example## :has-text(/\u200c/i)\n"
)
// urlFilterIDCounter is the atomic integer used to create unique filter IDs.
var urlFilterIDCounter = &atomic.Int32{}
// newURLFilterID returns a new unique URLFilterID.
func newURLFilterID() (id rulelist.URLFilterID) {
return rulelist.URLFilterID(urlFilterIDCounter.Add(1))
}
// newFilter is a helper for creating new filters in tests. It does not
// register the closing of the filter using t.Cleanup; callers must do that
// either directly or by using the filter in an engine.
func newFilter(t testing.TB, u *url.URL, name string) (f *rulelist.Filter) {
t.Helper()
f, err := rulelist.NewFilter(&rulelist.FilterConfig{
URL: u,
Name: name,
UID: rulelist.MustNewUID(),
URLFilterID: newURLFilterID(),
Enabled: true,
})
require.NoError(t, err)
return f
}
// newFilterLocations is a test helper that sets up both the filtering-rule list
// file and the HTTP-server. It also registers file removal and server stopping
// using t.Cleanup.
func newFilterLocations(
t testing.TB,
cacheDir string,
fileData string,
httpData string,
) (fileURL, srvURL *url.URL) {
filePath := filepath.Join(cacheDir, "initial.txt")
err := os.WriteFile(filePath, []byte(fileData), 0o644)
require.NoError(t, err)
testutil.CleanupAndRequireSuccess(t, func() (err error) {
return os.Remove(filePath)
})
fileURL = &url.URL{
Scheme: "file",
Path: filePath,
}
srv := newStringHTTPServer(httpData)
t.Cleanup(srv.Close)
srvURL, err = url.Parse(srv.URL)
require.NoError(t, err)
return fileURL, srvURL
}
// newStringHTTPServer returns a new HTTP server that serves s.
func newStringHTTPServer(s string) (srv *httptest.Server) {
return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {
pt := testutil.PanicT{}
_, err := io.WriteString(w, s)
require.NoError(pt, err)
}))
}

View file

@ -0,0 +1,98 @@
package rulelist
import (
"fmt"
"strings"
"sync"
"github.com/AdguardTeam/urlfilter"
"github.com/AdguardTeam/urlfilter/filterlist"
)
// TextEngine is a single DNS filter based on a list of rules in text form.
type TextEngine struct {
// mu protects engine and storage.
mu *sync.RWMutex
// engine is the filtering engine.
engine *urlfilter.DNSEngine
// storage is the filtering-rule storage. It is saved here to close it.
storage *filterlist.RuleStorage
// name is the human-readable name of the engine, like "custom".
name string
}
// TextEngineConfig is the configuration for a rule-list filtering engine
// created from a filtering rule text.
type TextEngineConfig struct {
// Name is the human-readable name of this engine, like "allowed",
// "blocked", or "custom".
Name string
// Rules is the text of the filtering rules for this engine.
Rules []string
// ID is the ID to use inside a URL-filter engine.
ID URLFilterID
}
// NewTextEngine returns a new rule-list filtering engine that uses rules
// directly. The engine is ready to use and should not be refreshed.
func NewTextEngine(c *TextEngineConfig) (e *TextEngine, err error) {
text := strings.Join(c.Rules, "\n")
storage, err := filterlist.NewRuleStorage([]filterlist.RuleList{
&filterlist.StringRuleList{
RulesText: text,
ID: c.ID,
IgnoreCosmetic: true,
},
})
if err != nil {
return nil, fmt.Errorf("creating rule storage: %w", err)
}
engine := urlfilter.NewDNSEngine(storage)
return &TextEngine{
mu: &sync.RWMutex{},
engine: engine,
storage: storage,
name: c.Name,
}, nil
}
// FilterRequest returns the result of filtering req using the DNS filtering
// engine.
func (e *TextEngine) FilterRequest(
req *urlfilter.DNSRequest,
) (res *urlfilter.DNSResult, hasMatched bool) {
var engine *urlfilter.DNSEngine
func() {
e.mu.RLock()
defer e.mu.RUnlock()
engine = e.engine
}()
return engine.MatchRequest(req)
}
// Close closes the underlying rule list engine as well as the rule lists.
func (e *TextEngine) Close() (err error) {
e.mu.Lock()
defer e.mu.Unlock()
if e.storage == nil {
return nil
}
err = e.storage.Close()
if err != nil {
return fmt.Errorf("closing text engine %q: %w", e.name, err)
}
return nil
}

View file

@ -0,0 +1,40 @@
package rulelist_test
import (
"testing"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/testutil"
"github.com/AdguardTeam/urlfilter"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func TestNewTextEngine(t *testing.T) {
eng, err := rulelist.NewTextEngine(&rulelist.TextEngineConfig{
Name: "RulesEngine",
Rules: []string{
testRuleTextTitle,
testRuleTextBlocked,
},
ID: testURLFilterID,
})
require.NoError(t, err)
require.NotNil(t, eng)
testutil.CleanupAndRequireSuccess(t, eng.Close)
fltReq := &urlfilter.DNSRequest{
Hostname: "blocked.example",
Answer: false,
DNSType: dns.TypeA,
}
fltRes, hasMatched := eng.FilterRequest(fltReq)
assert.True(t, hasMatched)
require.NotNil(t, fltRes)
require.NotNil(t, fltRes.NetworkRule)
assert.Equal(t, fltRes.NetworkRule.FilterListID, testURLFilterID)
}

View file

@ -14,6 +14,7 @@ import (
"time"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/cache"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter"
@ -98,7 +99,7 @@ func NewDefault(
cacheTTL: cacheTTL,
}
err = ss.resetEngine(filtering.SafeSearchListID, conf)
err = ss.resetEngine(rulelist.URLFilterIDSafeSearch, conf)
if err != nil {
// Don't wrap the error, because it's informative enough as is.
return nil, err
@ -234,7 +235,7 @@ func (ss *Default) newResult(
) (res *filtering.Result, err error) {
res = &filtering.Result{
Rules: []*filtering.ResultRule{{
FilterListID: filtering.SafeSearchListID,
FilterListID: rulelist.URLFilterIDSafeSearch,
}},
Reason: filtering.FilteredSafeSearch,
IsFiltered: true,
@ -368,7 +369,7 @@ func (ss *Default) Update(conf filtering.SafeSearchConfig) (err error) {
ss.mu.Lock()
defer ss.mu.Unlock()
err = ss.resetEngine(filtering.SafeSearchListID, conf)
err = ss.resetEngine(rulelist.URLFilterIDSafeSearch, conf)
if err != nil {
// Don't wrap the error, because it's informative enough as is.
return err

View file

@ -9,6 +9,7 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghtest"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/safesearch"
"github.com/AdguardTeam/golibs/testutil"
"github.com/miekg/dns"
@ -69,7 +70,7 @@ func TestDefault_CheckHost_yandex(t *testing.T) {
require.Len(t, res.Rules, 1)
assert.Equal(t, yandexIP, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
assert.Equal(t, rulelist.URLFilterIDSafeSearch, res.Rules[0].FilterListID)
}
}
@ -89,7 +90,7 @@ func TestDefault_CheckHost_yandexAAAA(t *testing.T) {
require.Len(t, res.Rules, 1)
assert.Empty(t, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
assert.Equal(t, rulelist.URLFilterIDSafeSearch, res.Rules[0].FilterListID)
}
func TestDefault_CheckHost_google(t *testing.T) {
@ -128,7 +129,7 @@ func TestDefault_CheckHost_google(t *testing.T) {
require.Len(t, res.Rules, 1)
assert.Equal(t, wantIP, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
assert.Equal(t, rulelist.URLFilterIDSafeSearch, res.Rules[0].FilterListID)
})
}
}
@ -180,7 +181,7 @@ func TestDefault_CheckHost_duckduckgoAAAA(t *testing.T) {
require.Len(t, res.Rules, 1)
assert.Empty(t, res.Rules[0].IP)
assert.EqualValues(t, filtering.SafeSearchListID, res.Rules[0].FilterListID)
assert.Equal(t, rulelist.URLFilterIDSafeSearch, res.Rules[0].FilterListID)
}
func TestDefault_Update(t *testing.T) {

View file

@ -11,6 +11,7 @@ import (
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/netutil"
"go.etcd.io/bbolt"
"golang.org/x/crypto/bcrypt"
)
@ -51,14 +52,15 @@ func (s *session) deserialize(data []byte) bool {
return true
}
// Auth - global object
// Auth is the global authentication object.
type Auth struct {
db *bbolt.DB
rateLimiter *authRateLimiter
sessions map[string]*session
users []webUser
lock sync.Mutex
sessionTTL uint32
trustedProxies netutil.SubnetSet
db *bbolt.DB
rateLimiter *authRateLimiter
sessions map[string]*session
users []webUser
lock sync.Mutex
sessionTTL uint32
}
// webUser represents a user of the Web UI.
@ -69,15 +71,22 @@ type webUser struct {
PasswordHash string `yaml:"password"`
}
// InitAuth - create a global object
func InitAuth(dbFilename string, users []webUser, sessionTTL uint32, rateLimiter *authRateLimiter) *Auth {
// InitAuth initializes the global authentication object.
func InitAuth(
dbFilename string,
users []webUser,
sessionTTL uint32,
rateLimiter *authRateLimiter,
trustedProxies netutil.SubnetSet,
) (a *Auth) {
log.Info("Initializing auth module: %s", dbFilename)
a := &Auth{
sessionTTL: sessionTTL,
rateLimiter: rateLimiter,
sessions: make(map[string]*session),
users: users,
a = &Auth{
sessionTTL: sessionTTL,
rateLimiter: rateLimiter,
sessions: make(map[string]*session),
users: users,
trustedProxies: trustedProxies,
}
var err error
a.db, err = bbolt.Open(dbFilename, 0o644, nil)
@ -95,7 +104,7 @@ func InitAuth(dbFilename string, users []webUser, sessionTTL uint32, rateLimiter
return a
}
// Close - close module
// Close closes the authentication database.
func (a *Auth) Close() {
_ = a.db.Close()
}
@ -104,7 +113,8 @@ func bucketName() []byte {
return []byte("sessions-2")
}
// load sessions from file, remove expired sessions
// loadSessions loads sessions from the database file and removes expired
// sessions.
func (a *Auth) loadSessions() {
tx, err := a.db.Begin(true)
if err != nil {
@ -156,7 +166,8 @@ func (a *Auth) loadSessions() {
log.Debug("auth: loaded %d sessions from DB (removed %d expired)", len(a.sessions), removed)
}
// store session data in file
// addSession adds a new session to the list of sessions and saves it in the
// database file.
func (a *Auth) addSession(data []byte, s *session) {
name := hex.EncodeToString(data)
a.lock.Lock()
@ -167,7 +178,7 @@ func (a *Auth) addSession(data []byte, s *session) {
}
}
// store session data in file
// storeSession saves a session in the database file.
func (a *Auth) storeSession(data []byte, s *session) bool {
tx, err := a.db.Begin(true)
if err != nil {

View file

@ -37,7 +37,7 @@ func TestAuth(t *testing.T) {
Name: "name",
PasswordHash: "$2y$05$..vyzAECIhJPfaQiOK17IukcQnqEgKJHy0iETyYqxn3YXJl8yZuo2",
}}
a := InitAuth(fn, nil, 60, nil)
a := InitAuth(fn, nil, 60, nil, nil)
s := session{}
user := webUser{Name: "name"}
@ -66,7 +66,7 @@ func TestAuth(t *testing.T) {
a.Close()
// load saved session
a = InitAuth(fn, users, 60, nil)
a = InitAuth(fn, users, 60, nil, nil)
// the session is still alive
assert.Equal(t, checkSessionOK, a.checkSession(sessStr))
@ -82,7 +82,7 @@ func TestAuth(t *testing.T) {
time.Sleep(3 * time.Second)
// load and remove expired sessions
a = InitAuth(fn, users, 60, nil)
a = InitAuth(fn, users, 60, nil, nil)
assert.Equal(t, checkSessionNotFound, a.checkSession(sessStr))
a.Close()

View file

@ -4,8 +4,8 @@ import (
"encoding/hex"
"encoding/json"
"fmt"
"net"
"net/http"
"net/netip"
"path"
"strconv"
"strings"
@ -78,7 +78,7 @@ func (a *Auth) newCookie(req loginJSON, addr string) (c *http.Cookie, err error)
// a well-maintained third-party module.
//
// TODO(a.garipov): Support header Forwarded from RFC 7329.
func realIP(r *http.Request) (ip net.IP, err error) {
func realIP(r *http.Request) (ip netip.Addr, err error) {
proxyHeaders := []string{
httphdr.CFConnectingIP,
httphdr.TrueClientIP,
@ -87,8 +87,8 @@ func realIP(r *http.Request) (ip net.IP, err error) {
for _, h := range proxyHeaders {
v := r.Header.Get(h)
ip = net.ParseIP(v)
if ip != nil {
ip, err = netip.ParseAddr(v)
if err == nil {
return ip, nil
}
}
@ -96,20 +96,20 @@ func realIP(r *http.Request) (ip net.IP, err error) {
// If none of the above yielded any results, get the leftmost IP address
// from the X-Forwarded-For header.
s := r.Header.Get(httphdr.XForwardedFor)
ipStrs := strings.SplitN(s, ", ", 2)
ip = net.ParseIP(ipStrs[0])
if ip != nil {
ipStr, _, _ := strings.Cut(s, ",")
ip, err = netip.ParseAddr(ipStr)
if err == nil {
return ip, nil
}
// When everything else fails, just return the remote address as understood
// by the stdlib.
ipStr, err := netutil.SplitHost(r.RemoteAddr)
ipStr, err = netutil.SplitHost(r.RemoteAddr)
if err != nil {
return nil, fmt.Errorf("getting ip from client addr: %w", err)
return netip.Addr{}, fmt.Errorf("getting ip from client addr: %w", err)
}
return net.ParseIP(ipStr), nil
return netip.ParseAddr(ipStr)
}
// writeErrorWithIP is like [aghhttp.Error], but includes the remote IP address
@ -142,8 +142,6 @@ func handleLogin(w http.ResponseWriter, r *http.Request) {
// to security issues.
//
// See https://github.com/AdguardTeam/AdGuardHome/issues/2799.
//
// TODO(e.burkov): Use realIP when the issue will be fixed.
if remoteIP, err = netutil.SplitHost(r.RemoteAddr); err != nil {
writeErrorWithIP(
r,
@ -173,20 +171,24 @@ func handleLogin(w http.ResponseWriter, r *http.Request) {
}
}
cookie, err := Context.auth.newCookie(req, remoteIP)
if err != nil {
writeErrorWithIP(r, w, http.StatusForbidden, remoteIP, "%s", err)
return
}
// Use realIP here, since this IP address is only used for logging.
ip, err := realIP(r)
if err != nil {
log.Error("auth: getting real ip from request with remote ip %s: %s", remoteIP, err)
}
log.Info("auth: user %q successfully logged in from ip %v", req.Name, ip)
cookie, err := Context.auth.newCookie(req, remoteIP)
if err != nil {
logIP := remoteIP
if Context.auth.trustedProxies.Contains(ip.Unmap()) {
logIP = ip.String()
}
writeErrorWithIP(r, w, http.StatusForbidden, logIP, "%s", err)
return
}
log.Info("auth: user %q successfully logged in from ip %s", req.Name, ip)
http.SetCookie(w, cookie)

View file

@ -1,8 +1,8 @@
package home
import (
"net"
"net/http"
"net/netip"
"net/textproto"
"net/url"
"path/filepath"
@ -39,7 +39,7 @@ func TestAuthHTTP(t *testing.T) {
users := []webUser{
{Name: "name", PasswordHash: "$2y$05$..vyzAECIhJPfaQiOK17IukcQnqEgKJHy0iETyYqxn3YXJl8yZuo2"},
}
Context.auth = InitAuth(fn, users, 60, nil)
Context.auth = InitAuth(fn, users, 60, nil, nil)
handlerCalled := false
handler := func(_ http.ResponseWriter, _ *http.Request) {
@ -125,13 +125,13 @@ func TestRealIP(t *testing.T) {
header http.Header
remoteAddr string
wantErrMsg string
wantIP net.IP
wantIP netip.Addr
}{{
name: "success_no_proxy",
header: nil,
remoteAddr: remoteAddr,
wantErrMsg: "",
wantIP: net.IPv4(1, 2, 3, 4),
wantIP: netip.MustParseAddr("1.2.3.4"),
}, {
name: "success_proxy",
header: http.Header{
@ -139,7 +139,7 @@ func TestRealIP(t *testing.T) {
},
remoteAddr: remoteAddr,
wantErrMsg: "",
wantIP: net.IPv4(1, 2, 3, 5),
wantIP: netip.MustParseAddr("1.2.3.5"),
}, {
name: "success_proxy_multiple",
header: http.Header{
@ -149,14 +149,14 @@ func TestRealIP(t *testing.T) {
},
remoteAddr: remoteAddr,
wantErrMsg: "",
wantIP: net.IPv4(1, 2, 3, 6),
wantIP: netip.MustParseAddr("1.2.3.6"),
}, {
name: "error_no_proxy",
header: nil,
remoteAddr: "1:::2",
wantErrMsg: `getting ip from client addr: address 1:::2: ` +
`too many colons in address`,
wantIP: nil,
wantIP: netip.Addr{},
}}
for _, tc := range testCases {

View file

@ -19,6 +19,7 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/whois"
"github.com/AdguardTeam/dnsproxy/proxy"
"github.com/AdguardTeam/dnsproxy/upstream"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/hostsfile"
"github.com/AdguardTeam/golibs/log"
@ -54,7 +55,7 @@ type clientsContainer struct {
// ipToRC maps IP addresses to runtime client information.
ipToRC map[netip.Addr]*client.Runtime
allTags *stringutil.Set
allTags *container.MapSet[string]
// dhcp is the DHCP service implementation.
dhcp DHCP
@ -108,7 +109,7 @@ func (clients *clientsContainer) Init(
clients.clientIndex = client.NewIndex()
clients.allTags = stringutil.NewSet(clientTags...)
clients.allTags = container.NewMapSet(clientTags...)
// TODO(e.burkov): Use [dhcpsvc] implementation when it's ready.
clients.dhcp = dhcpServer
@ -213,7 +214,7 @@ type clientObject struct {
// toPersistent returns an initialized persistent client if there are no errors.
func (o *clientObject) toPersistent(
filteringConf *filtering.Config,
allTags *stringutil.Set,
allTags *container.MapSet[string],
) (cli *client.Persistent, err error) {
cli = &client.Persistent{
Name: o.Name,
@ -307,8 +308,8 @@ func (clients *clientsContainer) forConfig() (objs []*clientObject) {
BlockedServices: cli.BlockedServices.Clone(),
IDs: cli.IDs(),
Tags: stringutil.CloneSlice(cli.Tags),
Upstreams: stringutil.CloneSlice(cli.Upstreams),
Tags: slices.Clone(cli.Tags),
Upstreams: slices.Clone(cli.Upstreams),
UID: cli.UID,

View file

@ -539,13 +539,13 @@ func fatalOnError(err error) {
// run configures and starts AdGuard Home.
func run(opts options, clientBuildFS fs.FS, done chan struct{}) {
// Configure config filename.
initConfigFilename(opts)
// Configure working dir and config path.
// Configure working dir.
err := initWorkingDir(opts)
fatalOnError(err)
// Configure config filename.
initConfigFilename(opts)
// Configure log level and output.
err = configureLogger(opts)
fatalOnError(err)
@ -674,8 +674,10 @@ func initUsers() (auth *Auth, err error) {
log.Info("authratelimiter is disabled")
}
trustedProxies := netutil.SliceSubnetSet(netutil.UnembedPrefixes(config.DNS.TrustedProxies))
sessionTTL := config.HTTPConfig.SessionTTL.Seconds()
auth = InitAuth(sessFilename, config.Users, uint32(sessionTTL), rateLimiter)
auth = InitAuth(sessFilename, config.Users, uint32(sessionTTL), rateLimiter, trustedProxies)
if auth == nil {
return nil, errors.Error("initializing auth module failed")
}
@ -758,11 +760,12 @@ func writePIDFile(fn string) bool {
}
// initConfigFilename sets up context config file path. This file path can be
// overridden by command-line arguments, or is set to default.
// overridden by command-line arguments, or is set to default. Must only be
// called after initializing the workDir with initWorkingDir.
func initConfigFilename(opts options) {
confPath := opts.confFilename
if confPath == "" {
Context.confFilePath = "AdGuardHome.yaml"
Context.confFilePath = filepath.Join(Context.workDir, "AdGuardHome.yaml")
return
}

View file

@ -5,12 +5,12 @@ import (
"net/http"
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
)
// TODO(a.garipov): Get rid of a global or generate from .twosky.json.
var allowedLanguages = stringutil.NewSet(
var allowedLanguages = container.NewMapSet(
"ar",
"be",
"bg",

View file

@ -271,11 +271,12 @@ func handleServiceCommand(s service.Service, action string, opts options) (err e
return fmt.Errorf("failed to run service: %w", err)
}
case "install":
initConfigFilename(opts)
if err = initWorkingDir(opts); err != nil {
return fmt.Errorf("failed to init working dir: %w", err)
}
initConfigFilename(opts)
handleServiceInstallCommand(s)
case "uninstall":
handleServiceUninstallCommand(s)

View file

@ -9,6 +9,7 @@ import (
"strings"
"sync"
"github.com/AdguardTeam/golibs/container"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/digineo/go-ipset/v2"
@ -174,18 +175,6 @@ func (p *props) parseAttrData(a netfilter.Attribute) {
}
}
// unit is a convenient alias for struct{}.
type unit = struct{}
// ipsInIpset is the type of a set of IP-address-to-ipset mappings.
type ipsInIpset map[ipInIpsetEntry]unit
// ipInIpsetEntry is the type for entries in an ipsInIpset set.
type ipInIpsetEntry struct {
ipsetName string
ipArr [net.IPv6len]byte
}
// manager is the Linux Netfilter ipset manager.
type manager struct {
nameToIpset map[string]props
@ -196,17 +185,24 @@ type manager struct {
// mu protects all properties below.
mu *sync.Mutex
// TODO(a.garipov): Currently, the ipset list is static, and we don't
// read the IPs already in sets, so we can assume that all incoming IPs
// are either added to all corresponding ipsets or not. When that stops
// being the case, for example if we add dynamic reconfiguration of
// ipsets, this map will need to become a per-ipset-name one.
addedIPs ipsInIpset
// TODO(a.garipov): Currently, the ipset list is static, and we don't read
// the IPs already in sets, so we can assume that all incoming IPs are
// either added to all corresponding ipsets or not. When that stops being
// the case, for example if we add dynamic reconfiguration of ipsets, this
// map will need to become a per-ipset-name one.
addedIPs *container.MapSet[ipInIpsetEntry]
ipv4Conn ipsetConn
ipv6Conn ipsetConn
}
// ipInIpsetEntry is the type for entries in [manager.addIPs].
type ipInIpsetEntry struct {
ipsetName string
// TODO(schzen): Use netip.Addr.
ipArr [net.IPv6len]byte
}
// dialNetfilter establishes connections to Linux's netfilter module.
func (m *manager) dialNetfilter(conf *netlink.Config) (err error) {
// The kernel API does not actually require two sockets but package
@ -372,7 +368,7 @@ func newManagerWithDialer(ipsetConf []string, dial dialer) (mgr Manager, err err
dial: dial,
addedIPs: make(ipsInIpset),
addedIPs: container.NewMapSet[ipInIpsetEntry](),
}
err = m.dialNetfilter(&netlink.Config{})
@ -438,7 +434,7 @@ func (m *manager) addIPs(host string, set props, ips []net.IP) (n int, err error
}
copy(e.ipArr[:], ip.To16())
if _, added := m.addedIPs[e]; added {
if m.addedIPs.Has(e) {
continue
}
@ -471,7 +467,7 @@ func (m *manager) addIPs(host string, set props, ips []net.IP) (n int, err error
for _, e := range newAddedEntries {
s := m.nameToIpset[e.ipsetName]
if s.isPersistent {
m.addedIPs[e] = unit{}
m.addedIPs.Add(e)
}
}

View file

@ -11,6 +11,7 @@ import (
"time"
"github.com/AdguardTeam/AdGuardHome/internal/filtering"
"github.com/AdguardTeam/AdGuardHome/internal/filtering/rulelist"
"github.com/AdguardTeam/golibs/errors"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/urlfilter/rules"
@ -179,7 +180,8 @@ func decodeResultRuleKey(key string, i int, dec *json.Decoder, ent *logEntry) {
case "FilterListID":
ent.Result.Rules, vToken = decodeVTokenAndAddRule(key, i, dec, ent.Result.Rules)
if n, ok := vToken.(json.Number); ok {
ent.Result.Rules[i].FilterListID, _ = n.Int64()
id, _ := n.Int64()
ent.Result.Rules[i].FilterListID = rulelist.URLFilterID(id)
}
case "IP":
ent.Result.Rules, vToken = decodeVTokenAndAddRule(key, i, dec, ent.Result.Rules)
@ -582,7 +584,7 @@ var resultHandlers = map[string]logEntryHandler{
return nil
}
i, err := n.Int64()
id, err := n.Int64()
if err != nil {
return err
}
@ -593,7 +595,7 @@ var resultHandlers = map[string]logEntryHandler{
l++
}
ent.Result.Rules[l-1].FilterListID = i
ent.Result.Rules[l-1].FilterListID = rulelist.URLFilterID(id)
return nil
},

View file

@ -7,6 +7,7 @@ import (
"net"
"net/http"
"net/url"
"slices"
"strconv"
"strings"
"time"
@ -15,7 +16,6 @@ import (
"github.com/AdguardTeam/AdGuardHome/internal/aghhttp"
"github.com/AdguardTeam/AdGuardHome/internal/aghnet"
"github.com/AdguardTeam/golibs/log"
"github.com/AdguardTeam/golibs/stringutil"
"github.com/AdguardTeam/golibs/timeutil"
"golang.org/x/net/idna"
)
@ -308,7 +308,7 @@ func parseSearchCriterion(q url.Values, name string, ct criterionType) (
asciiVal = ""
}
case ctFilteringStatus:
if !stringutil.InSlice(filteringStatusValues, val) {
if !slices.Contains(filteringStatusValues, val) {
return false, sc, fmt.Errorf("invalid value %s", val)
}
default:

View file

@ -26,7 +26,7 @@ require (
github.com/kyoh86/nolint v0.0.1 // indirect
github.com/xo/terminfo v0.0.0-20220910002029-abceb7e1c41e // indirect
golang.org/x/exp v0.0.0-20230321023759-10a507213a29 // indirect
golang.org/x/exp/typeparams v0.0.0-20240222234643-814bf88cf225 // indirect
golang.org/x/exp/typeparams v0.0.0-20240325151524-a685a6edb6d8 // indirect
golang.org/x/mod v0.16.0 // indirect
golang.org/x/sync v0.6.0 // indirect
golang.org/x/sys v0.18.0 // indirect

View file

@ -63,8 +63,8 @@ golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPh
golang.org/x/crypto v0.0.0-20210921155107-089bfa567519/go.mod h1:GvvjBRRGRdwPK5ydBHafDWAxML/pGHZbMvKqRZ5+Abc=
golang.org/x/exp v0.0.0-20230321023759-10a507213a29 h1:ooxPy7fPvB4kwsA2h+iBNHkAbp/4JxTSwCmvdjEYmug=
golang.org/x/exp v0.0.0-20230321023759-10a507213a29/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
golang.org/x/exp/typeparams v0.0.0-20240222234643-814bf88cf225 h1:BzKNaIRXh1bD+1557OcFIHlpYBiVbK4zEyn8zBHi1SE=
golang.org/x/exp/typeparams v0.0.0-20240222234643-814bf88cf225/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/exp/typeparams v0.0.0-20240325151524-a685a6edb6d8 h1:ShhqwXlNzuDeQzaa6htzo1S333ACXZzJZgZLpKAza8E=
golang.org/x/exp/typeparams v0.0.0-20240325151524-a685a6edb6d8/go.mod h1:AbB0pIl9nAr9wVwH+Z2ZpaocVmF5I4GyWCDIsVjR0bk=
golang.org/x/mod v0.3.0/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
golang.org/x/mod v0.6.0-dev.0.20220419223038-86c51ed26bb4/go.mod h1:jJ57K6gSWd91VN4djpZkiMVwK6gcyfeH4XE8wZrZaV4=

View file

@ -4,6 +4,10 @@ package main
import (
"embed"
// Embed tzdata in binary.
//
// See https://github.com/AdguardTeam/AdGuardHome/issues/6758
_ "time/tzdata"
"github.com/AdguardTeam/AdGuardHome/internal/home"
)

View file

@ -52,7 +52,7 @@ func prepareMultipartMsg(
w := multipart.NewWriter(buf)
var fw io.Writer
err = mapsutil.OrderedRangeError(formData, w.WriteField)
err = mapsutil.SortedRangeError(formData, w.WriteField)
if err != nil {
return nil, "", fmt.Errorf("writing field: %w", err)
}