What is the difference between the big and small lists? 🔗
The big list (oisd big) blocks: Ads, (Mobile) App Ads, Phishing, Malvertising, Malware, Spyware, Ransomware, CryptoJacking, Scam
... Telemetry/Analytics/Tracking (Where not needed for proper functionality)
The small list (oisd small) is a smaller, less comprehensive variant of the big list, which focuses mainly on Ads, (Mobile) App Ads
The NSFW list (oisd NSFW, which is also a big list) blocks Shock/Porn/Adult domains
The small NSFW list (oisd NSFW small) blocks only the NSFW domains found in the Tranco List (top 1 million domains)
The small list (oisd small) is a smaller, less comprehensive variant of the big list, which focuses mainly on Ads, (Mobile) App Ads
The NSFW list (oisd NSFW, which is also a big list) blocks Shock/Porn/Adult domains
The small NSFW list (oisd NSFW small) blocks only the NSFW domains found in the Tranco List (top 1 million domains)
How often is the list updated? 🔗
The quick answer is; At least once every 24 hours.
It really depends on if there are changes in the source lists, and when false positives get removed from the oisd list.
A good guess from me would be:
oisd big: at least once every 24 hours
oisd small: updated daily, but don't be surprised if there are some days it's not
It really depends on if there are changes in the source lists, and when false positives get removed from the oisd list.
A good guess from me would be:
oisd big: at least once every 24 hours
oisd small: updated daily, but don't be surprised if there are some days it's not
Are there any known issues with using oisd? 🔗
Yes. Nothing is perfect.
- Websites showing "Please disable adblocker" (Would not make sense to fix that, as allowing the domain to fix that would most probably cause ads to be allowed on the majority of other websites.)
- Advertising-domains main websites (Most of the time, ads are not loaded from their main domains, but rather subdomains. But as the main goal of this blocklist is to prevent ads, and adding the main domain automatically blocks all subdomains they will not be removed.
- Mobile games; likely to not load user requested video ads for extra's / double points
- Google Analytics dashboard
Needs analytics.google.com, yet this is blocked because I see a lot of requests going out to region1.analytics.google.com without using the dashboard.
pagead2.googlesyndication.com is needed for;
- Dark mode @ 9to5mac.com
- ubitennis.com
securepubads.g.doubleclick.net is needed for;
- Trailer playback @ allocine.fr
- Video playback @ mirror.co.uk
- servustv.com
- whatismyip.com (along with blocked cdn-a.yieldlove.com)
- Podcast playback @ femmeactuelle.fr
- Video playback @ goplay.be
- cinemaxx.de site layout
- Video playback @ packhacker.com
- Video playback @ moviepilot.de
- Video playback @ filmstarts.de
- Video playback @ www.thestreet.com
- Video playback @ independent.co.uk
- Bloomberg TV in iOS ?
- Bloomberg app @ Android - Media
pubads.g.doubleclick.net is needed for;
- Video playback @ https://www.cbsnews.com/live/
*.g.doubleclick.net is probably the strongest adblocking rule there is. It will remain blocked.
If one really needs them whitelisted, this will be up to the user to add an exception on their end.
- Websites showing "Please disable adblocker" (Would not make sense to fix that, as allowing the domain to fix that would most probably cause ads to be allowed on the majority of other websites.)
- Advertising-domains main websites (Most of the time, ads are not loaded from their main domains, but rather subdomains. But as the main goal of this blocklist is to prevent ads, and adding the main domain automatically blocks all subdomains they will not be removed.
- Mobile games; likely to not load user requested video ads for extra's / double points
- Google Analytics dashboard
Needs analytics.google.com, yet this is blocked because I see a lot of requests going out to region1.analytics.google.com without using the dashboard.
pagead2.googlesyndication.com is needed for;
- Dark mode @ 9to5mac.com
- ubitennis.com
securepubads.g.doubleclick.net is needed for;
- Trailer playback @ allocine.fr
- Video playback @ mirror.co.uk
- servustv.com
- whatismyip.com (along with blocked cdn-a.yieldlove.com)
- Podcast playback @ femmeactuelle.fr
- Video playback @ goplay.be
- cinemaxx.de site layout
- Video playback @ packhacker.com
- Video playback @ moviepilot.de
- Video playback @ filmstarts.de
- Video playback @ www.thestreet.com
- Video playback @ independent.co.uk
- Bloomberg TV in iOS ?
- Bloomberg app @ Android - Media
pubads.g.doubleclick.net is needed for;
- Video playback @ https://www.cbsnews.com/live/
*.g.doubleclick.net is probably the strongest adblocking rule there is. It will remain blocked.
If one really needs them whitelisted, this will be up to the user to add an exception on their end.
Why is oisd no longer providing the oisd blocklists in domains and hosts formats? 🔗
First let's look at the domain blocking possibilities:
A) Domains: baddomain.org
B) www.-prefixed domains: www.baddomain.org
C) Known subdomains: badsubdomain.baddomain.org
D) (yet) unknown subdomains: badunknownsub.baddomain.org
E) Randomly generated subdomains: bad239ue9f59gw.baddomain.org
Using HOSTS or DOMAINS syntax, you can block:
A) The domain (baddomain.org) with just one rule.
B) The www.-prefixed domain (www.baddomain.org) with an additional rule.
C) A known subdomain (badsubdomain.baddomain.org) with an added rule per each subdomain.
However, using HOSTS or DOMAINS syntax, it becomes impossible to block:
D) Unknown subdomains (e.g., badunknownsub.baddomain.org) because they need to be individually listed in advance.
E) Randomly generated subdomains (e.g., bad239ue9f59gw.baddomain.org) as it is not feasible to predict or list all possible variations.
In contrast, by using a syntax like ABP, DNSmasq, Domains (wildcards), or Regex:
A) You can block the main domain (baddomain.org) with just one rule.
B) This same rule will also block the www.-prefixed domain (www.baddomain.org).
C) All known subdomains (badsubdomain.baddomain.org) are also be blocked using that same rule.
D) Even for unknown subdomains, they will effectively be blocked with that same rule.
E) Similarly, randomly generated subdomains will also successfully be blocked using that same rule.
When providing/building HOSTS / Domains syntaxes, I gathered all the subdomains available in known blocking lists and included them in the blocklist (filtered to exclude NXDOMAINS and SERVFAILS). While this may seem like a logical step, it also presented a certain level of illogicality and inefficiency. Effectively, blocklists using ABP, DNSmasq, Domains (wildcards), Regex -syntaxes are about 4 times smaller than HOSTS / Domains -syntaxes, AND block MORE. This is not something new, and has been known for years.
A) Domains: baddomain.org
B) www.-prefixed domains: www.baddomain.org
C) Known subdomains: badsubdomain.baddomain.org
D) (yet) unknown subdomains: badunknownsub.baddomain.org
E) Randomly generated subdomains: bad239ue9f59gw.baddomain.org
Using HOSTS or DOMAINS syntax, you can block:
A) The domain (baddomain.org) with just one rule.
B) The www.-prefixed domain (www.baddomain.org) with an additional rule.
C) A known subdomain (badsubdomain.baddomain.org) with an added rule per each subdomain.
However, using HOSTS or DOMAINS syntax, it becomes impossible to block:
D) Unknown subdomains (e.g., badunknownsub.baddomain.org) because they need to be individually listed in advance.
E) Randomly generated subdomains (e.g., bad239ue9f59gw.baddomain.org) as it is not feasible to predict or list all possible variations.
In contrast, by using a syntax like ABP, DNSmasq, Domains (wildcards), or Regex:
A) You can block the main domain (baddomain.org) with just one rule.
B) This same rule will also block the www.-prefixed domain (www.baddomain.org).
C) All known subdomains (badsubdomain.baddomain.org) are also be blocked using that same rule.
D) Even for unknown subdomains, they will effectively be blocked with that same rule.
E) Similarly, randomly generated subdomains will also successfully be blocked using that same rule.
When providing/building HOSTS / Domains syntaxes, I gathered all the subdomains available in known blocking lists and included them in the blocklist (filtered to exclude NXDOMAINS and SERVFAILS). While this may seem like a logical step, it also presented a certain level of illogicality and inefficiency. Effectively, blocklists using ABP, DNSmasq, Domains (wildcards), Regex -syntaxes are about 4 times smaller than HOSTS / Domains -syntaxes, AND block MORE. This is not something new, and has been known for years.
What does: "This list might be too big." mean? 🔗
This list is not verified to work (with the software in question).
It might work correctly.
It might not work correctly.
It might not work at all.
Some software might show a warning, yet it will work correctly...
Some might not show any indication of it being too big while it won't be working correctly
Some software will truncate the list (undesirable)
It might even cause problems (high MEM/CPU use, crashes/hangs, etc).
Please use at your own discretion
It might work correctly.
It might not work correctly.
It might not work at all.
Some software might show a warning, yet it will work correctly...
Some might not show any indication of it being too big while it won't be working correctly
Some software will truncate the list (undesirable)
It might even cause problems (high MEM/CPU use, crashes/hangs, etc).
Please use at your own discretion
How often is the list pruned of domains that no longer exist? 🔗
Every 5 days
The list(s) may contain domains which by themself do not resolve (example: ad.xiaomi.com), but it's subdomains DO resolve (eg; zeus.ad.xiaomi.com, track.ad.xiaomi.com, sdkconfig.ad.xiaomi.com, log.ad.xiaomi.com, globalapi.ad.xiaomi.com, e.ad.xiaomi.com, api.ad.xiaomi.com).
So by listing only one domain (even if it doesn't resolve itself), it blocks access to all subdomains, so they don't have to be listed.
The list(s) may contain domains which by themself do not resolve (example: ad.xiaomi.com), but it's subdomains DO resolve (eg; zeus.ad.xiaomi.com, track.ad.xiaomi.com, sdkconfig.ad.xiaomi.com, log.ad.xiaomi.com, globalapi.ad.xiaomi.com, e.ad.xiaomi.com, api.ad.xiaomi.com).
So by listing only one domain (even if it doesn't resolve itself), it blocks access to all subdomains, so they don't have to be listed.
I have a domain that should be listed, can it be added to oisd? 🔗
Quite possibly
I stopped manually adding domains to small/big as it is too time consuming and quite risky (I don't want to be the cause of breakage), but you can contact any of the included lists and have them include it. Once they include it should get included in oisd as well.
On the other hand; If you know of any NSFW domains not blocked by the oisd nsfw list, you can use this form to report/add them.
I stopped manually adding domains to small/big as it is too time consuming and quite risky (I don't want to be the cause of breakage), but you can contact any of the included lists and have them include it. Once they include it should get included in oisd as well.
On the other hand; If you know of any NSFW domains not blocked by the oisd nsfw list, you can use this form to report/add them.
I want to check in which list(s) a domain included in oisd originates, is that possible? 🔗
Yes, just use the false positive report form and type 'test' as the 'reason'. The report will then be ignored.
Why is list "X" not included? 🔗
If you find a blocklist that known to me, but not included;
That is because it doesn't pass the oisd-"inclusion test".
Reasons for this;
- It's a combination of other blocklists, or
- it's too old and no longer maintained, or
- it's a duplicate (or mirror) of another list, or
- it malformed and doesn't make sense, or
- it contains too many false positives, or
- it contains one (or more) domains that blocks something that should never be blocked, or
- it's syntax is not supported (yet).
Most of the time the reason why a blocklist is not included into oisd is because "it contains one (or more) domains that blocks something that should never be blocked".
Here are some examples; apple.com, microsoft.com, google.com, facebook.com, amazon.com, login.live.com, youtube.com, reddit.com, adidas.com ... you get the point.
There's a reason as to why I won't be disclosing the full list of those domains;
Some years ago, a blocklist maintainer asked me why his list wasn't included. (It wasn't included, because it included a domain as stated above).
Of course I was willing to help, and told him all the issues (that I was aware of) regarding his list.
He fixed them all, and as a result, his blocklist got included in the oisd list.
Within hours, false positive reports rolled in, all pointing to his list... and I was forced to again exclude it.
So.. lesson learned. If a list is "that bad", it most probably has way more bad entries (that I don't know of yet), and letting them know the issues (that I'm currently aware of) to get it included, will have a too high risk of introducing lot's of false positives into oisd.
Sidenote:
All non-included blocklists because of this ARE re-checked daily, so if their reasons for not being included DO get fixed (by other means of ME letting them know), they will get included automatically.
That is because it doesn't pass the oisd-"inclusion test".
Reasons for this;
- It's a combination of other blocklists, or
- it's too old and no longer maintained, or
- it's a duplicate (or mirror) of another list, or
- it malformed and doesn't make sense, or
- it contains too many false positives, or
- it contains one (or more) domains that blocks something that should never be blocked, or
- it's syntax is not supported (yet).
Most of the time the reason why a blocklist is not included into oisd is because "it contains one (or more) domains that blocks something that should never be blocked".
Here are some examples; apple.com, microsoft.com, google.com, facebook.com, amazon.com, login.live.com, youtube.com, reddit.com, adidas.com ... you get the point.
There's a reason as to why I won't be disclosing the full list of those domains;
Some years ago, a blocklist maintainer asked me why his list wasn't included. (It wasn't included, because it included a domain as stated above).
Of course I was willing to help, and told him all the issues (that I was aware of) regarding his list.
He fixed them all, and as a result, his blocklist got included in the oisd list.
Within hours, false positive reports rolled in, all pointing to his list... and I was forced to again exclude it.
So.. lesson learned. If a list is "that bad", it most probably has way more bad entries (that I don't know of yet), and letting them know the issues (that I'm currently aware of) to get it included, will have a too high risk of introducing lot's of false positives into oisd.
Sidenote:
All non-included blocklists because of this ARE re-checked daily, so if their reasons for not being included DO get fixed (by other means of ME letting them know), they will get included automatically.
I'm using the oisd blocklist but I still see ads? 🔗
There can be different reasons for this:
- Maybe the ads are 'baked-in' (Seen in some mobile apps/games) (Baked-in ads cannot be blocked)
- Maybe the domain is also being used for legit (non-ad) content (Blocking the domain would also block the legit content.)
Also read; Do you have any other recommendations besides this blocklist?.
- Maybe the ads are 'baked-in' (Seen in some mobile apps/games) (Baked-in ads cannot be blocked)
- Maybe the domain is also being used for legit (non-ad) content (Blocking the domain would also block the legit content.)
Also read; Do you have any other recommendations besides this blocklist?.
Do you report false positives upstream? 🔗
No. With the amount of lists included, that's near impossible.
The reasons for removal are also often not in line with the original blocklist.
False postives (and 'dead' domains) are publicly viewable per-list on the 'included lists'-page, so if you are a blocklist maintainer (and your list is being included in oisd), or you are just curious, it's easy for to check which domains are not being used by oisd, and why ;)
The reasons for removal are also often not in line with the original blocklist.
False postives (and 'dead' domains) are publicly viewable per-list on the 'included lists'-page, so if you are a blocklist maintainer (and your list is being included in oisd), or you are just curious, it's easy for to check which domains are not being used by oisd, and why ;)
Should I leave the default lists enabled? 🔗
When using this list in browser add-on blockers (eg. uBlock Origin, Safari Filters);
Yes definitely, as this is not meant to be a replacement for it's default lists.
When using this list in network based DNS blockers (eg Pi-hole, AdGuard Home); I'd say NO
Where the default lists are known to break stuff (shopping referral links, google shopping, video playback, etc), the whole idea of this blocklist is to have ZERO breakage, That's why if you share my opinion on having a functional internet experience as first priority and an ad/tracker/etc blocking as second, the answer is NO.
If you don't share that opinion, the answer will be YES, please DO keep the defaults enabled.
Yes definitely, as this is not meant to be a replacement for it's default lists.
When using this list in network based DNS blockers (eg Pi-hole, AdGuard Home); I'd say NO
Where the default lists are known to break stuff (shopping referral links, google shopping, video playback, etc), the whole idea of this blocklist is to have ZERO breakage, That's why if you share my opinion on having a functional internet experience as first priority and an ad/tracker/etc blocking as second, the answer is NO.
If you don't share that opinion, the answer will be YES, please DO keep the defaults enabled.
Does oisd block youtube ads? 🔗
No. Youtube ads cannot be blocked by DNS based blockers (where oisd is made for)
Adguard article
pi-hole.net topic
Alternatives:
- Use your browser, with add-on adblocker like uBlock Origin (Desktop, mobile)
- Newpipe (Android)
- ReVanced (Android)
- https://kb.adguard.com/en/android/solving-problems/youtube-ads (Android)
- Youtube Premium.
So as impossible as you now know they say it is, there still exist a bunch of lists that DO claim to block youtube ads. I'll list them here so you can try them as you please;
https://raw.githubusercontent.com/Ewpratten/youtube_ad_blocklist/master/blocklist.txt
https://raw.githubusercontent.com/CitizenXVIL/Hosts/master/Youtube%20hosts.txt
https://raw.githubusercontent.com/LaurentFough/pihole-youtube-adblock/master/pihole-youtube-adblock.txt
https://raw.githubusercontent.com/Rafaelrico77/piholeyt/master/youtubeadblock.txt
https://raw.githubusercontent.com/RootFiber/youtube-ads/main/ad-block-YouTube-Project.txt
https://raw.githubusercontent.com/RootFiber/youtube-ads/main/blockeverything.txt
https://raw.githubusercontent.com/anudeepND/youtubeadsblacklist/master/domainlist.txt
https://adblock.amroemischengutshof.de/youtube-ads-list.txt
https://blocklistproject.github.io/Lists/youtube.txt
https://framagit.org/PoorPocketsMcNewHold/no-google/raw/master/categories/youtube.txt
https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Formats/GoodbyeAds-YouTube-AdBlock-Filter.txt
https://raw.githubusercontent.com/kboghdady/youTube_ads_4_pi-hole/master/black.list
Adguard article
pi-hole.net topic
Alternatives:
- Use your browser, with add-on adblocker like uBlock Origin (Desktop, mobile)
- Newpipe (Android)
- ReVanced (Android)
- https://kb.adguard.com/en/android/solving-problems/youtube-ads (Android)
- Youtube Premium.
So as impossible as you now know they say it is, there still exist a bunch of lists that DO claim to block youtube ads. I'll list them here so you can try them as you please;
https://raw.githubusercontent.com/Ewpratten/youtube_ad_blocklist/master/blocklist.txt
https://raw.githubusercontent.com/CitizenXVIL/Hosts/master/Youtube%20hosts.txt
https://raw.githubusercontent.com/LaurentFough/pihole-youtube-adblock/master/pihole-youtube-adblock.txt
https://raw.githubusercontent.com/Rafaelrico77/piholeyt/master/youtubeadblock.txt
https://raw.githubusercontent.com/RootFiber/youtube-ads/main/ad-block-YouTube-Project.txt
https://raw.githubusercontent.com/RootFiber/youtube-ads/main/blockeverything.txt
https://raw.githubusercontent.com/anudeepND/youtubeadsblacklist/master/domainlist.txt
https://adblock.amroemischengutshof.de/youtube-ads-list.txt
https://blocklistproject.github.io/Lists/youtube.txt
https://framagit.org/PoorPocketsMcNewHold/no-google/raw/master/categories/youtube.txt
https://raw.githubusercontent.com/jerryn70/GoodbyeAds/master/Formats/GoodbyeAds-YouTube-AdBlock-Filter.txt
https://raw.githubusercontent.com/kboghdady/youTube_ads_4_pi-hole/master/black.list
Is there a list of domains that are being actively excluded? 🔗
Yes. Domains that are listed in included blocklists but are excluded from oisd, can be found here.
Will you open source the project? 🔗
I combine lists, remove domains that do not resolve, remove false positives, and the remainder is the list.
It's a list of domains, not software which potentially could have a backdoor.
Let's not overcomplicate things.
It's a list of domains, not software which potentially could have a backdoor.
Let's not overcomplicate things.
Do you have any other recommendations besides this blocklist? 🔗
Yes. This is a domain blocklist, designed to be used in a network wide DNS based ad-blocker (Eg. Adguard Home, Pi-hole). Since it cannot block youtube ads, hide ad-placeholders, or ads being served from the same domain as the wanted content, a browser add-on ad-blocker (Eg. uBlock Origin or the AdGuard Browser extension) is recommended.
Under which license are the oisd lists distributed? 🔗
What does oisd stand for? 🔗
It's not an abbreviation, it's simply a few characters bumped together. It works. You're here right?
How can I contact the oisd maintainer? 🔗
Found a bug? Have a question or recommendation? Something else? Contact me at;
