Privacy redirector json created for pattern checker module

[Raj9039852537]

privacy redirector extension support is still pending and I have managed to make it work with pattern checker module. Here is the link of my configuration for the same privacy redirector in json formal as required by pattern checker.

For temporary or permanent solution this can be used for redirecting the sites.
Feel free to use and edit the gist mentioned according to the needs

[TrianguloY]

This is exactly what I had in mind while adding the advanced json editor, letting users create their own patterns in a (relatively) easy way. Good job!

I, too, have a few personal patterns myself. I now need to think of a way to allow users to discover and upload/download them...

Maybe it's time to setup the Github wiki 🤔

[Raj9039852537]

Maybe it's time to setup the Github wiki 🤔

That will be more useful for the project

[Ilithy]

it is not possible to integrate the Json file into Url Checker, and let users choose to activate or not such and such a redirection?
By this I mean providing a maximum of alternatives directly in the Json file (for example, for Youtube, "providing" Invidious, but also NewPipe, and others), and providing the choice to activate them or not?
The file might be updated (by the developer, or by the community via PR) during Url Checker updates. And "will not force" users to relatively technical manipulations (ordinary users).

Am I delirious or is this possible?

[TrianguloY]

It is indeed possible, and I though about adding them disabled by default. But the names are in english, and having them translatable is the real pain...
A good solution would probably be allowing different urls of remote files with the rules (like the ClearURLs catalog but with multiple), but of course the work required for that is not trivial. Time to time.
In the meantime, copy/paste always work (I may add a link to the wiki when it is configured though, for easy access).

[Ilithy]

I can't imagine the amount of work that this entails, I have every confidence in you on this subject.

You are right, it is easier to make it functional by use your idea (provide the Json, and the instructions for modifying the rules, as for ClearUrl), my intervention was clumsy.

On the other hand, if you talk about the names of the various applications, they do not have to be translated (in the majority of cases, the name of the application remains the same, regardless of the language of the user)

[TrianguloY]

Hmm, that's right, I can add just 'Youtube -> Invidious'. I'll probably do it.
In any case, a wiki is also needed.

[Raj9039852537]

possible json schema for multi instance links like invidious would be
"Youtube to Invidious": {
"regex": "^https?:\\\/\\\/(www\\.|m\\.)?(youtube\\.com|youtu\\.be|youtube-nocookie\\.com)\\\/(.*)",
"replacement_urls": ["yewtu.be","invidious.snopyta.org"]
"selected_replacement": "1"
"replacement": "https:\/\/"+ selected_replacement +"\/$1"
}

where replacements_urls contains list of some/all instances
and selected_replacement will contain index of selected instance
and then replacement string will take into account that particular instance while applying the rule.
btw I don't know much java myself.

Also made some changes to fix escaping of '.' character not previously made (UrlChecker not responding due to this improper escaping) and added some more sites to the gist.

[TrianguloY]

As you probably saw, I (finally) created a wiki for user-patterns: https://github.com/TrianguloY/UrlChecker/wiki/Custom-patterns
I also moved this issue as discussion, I think it fits better here.

As for the app, I'll try to include a couple ones (disabled by default) for those that are interested to be able to discover them. In any case, I'll also add a link to that new wiki page, so this is just a 'sample'.

---

Question: If you need to choose a few of them, which ones do you think are the most 'representative' of privacy redirectors? Preferably some that will most likely work in the future and as popular as possible.

I was thinking about:

• reddit.com --> teddit.net
• twitter.com --> nitter.net
• youtube.com --> yewtu.be

[Raj9039852537]

Privacy Redirector by default provides multiple options and allow users to choose from amongst them.
I propose using similar architecture just like the code i posted on 24 aug in this thread.
for replacement_urls provide list of arrays containing all alternatives and then allow user to change selected-replacement string to their desired instance.( 0 to last index item)

Also provide farside links ( i think last item of array for this ) when available so as to randomized behaviour if someone want that behaviour.
For default instances the ones you listed are good.
Also since there is already disable option with each item , you can add them disabled by default .

Also I have updated the list yesterday to my current used one at gist

[TrianguloY]

For the next version:

• I added the random setting. Now 'replacement' can be a string (as before) or a list of strings. In that case, a random element from the list will be chosen each time you apply the rule.
• I added those three samples (reddit/twitter/youtube) disabled by default. The youtube has yewtube and farside.
  
• As I said there is now a link to the wiki, your gist is currently the only entry, but it's a very powerful one 👍

Hope this is enough to at least allow users to configure as desired. I know it's currently very manual, but hopefully it's customizable. I will consider adding a separate privacy redirector module in the future, but for now this should do.

Note: For multiple options to choose you can add multiple entries, as long as each entry has a different name they will be all shown with independent apply buttons (also, if you want to have multiple regexp, you can either have multiple entries too or use a "(a)|(b)" regexp).

[Ilithy]

It doesn't work either (the search keyword is not copied, it just sends me to metasearch.nl), don't worry, this is not the end of the world.
Thanks for taking a look and thanks for your kindness

[TrianguloY]

Maybe this?

"Google to searxNG": {
    "regex": "^https?:\/\/([a-z]+\\.)*?google\\.[a-z]+\/search\\?(.*&)*(q=[^&]*).*",
    "replacement": "https:\/\/metasearch.nl\/search?$3&category_images=on",
    "automatic": "true"
  }

[Ilithy]

Maybe this?

"Google to searxNG": {
    "regex": "^https?:\/\/([a-z]+\\.)*?google\\.[a-z]+\/search\\?(.*&)*(q=[^&]*).*",
    "replacement": "https:\/\/metasearch.nl\/search?$3&category_images=on",
    "automatic": "true"
  }

😲 Wow, find on first try and it works perfectly ✅, thanks ❤️

And thank you very much to both of you, it's really great.

thanks again.

[TrianguloY]

Happy to hear that :)
But not a first try at all! It's another iteration of Raj's samples with things that they may not work for you as expected. They should have work, but they are maybe too generic so I tweaked it to be more specific.

So, thanks to Raj!

[Ilithy]

Ok I see, and on closer inspection indeed @Raj9039852537 was not far from succeeding.
@Raj9039852537's motivation to help me commands respect; thanks again to both of you.

Apart, I did several tests to see if I could put this in default, but no, even using several keywords, it works.

[Ilithy]

Hi @TrianguloY how to share new "customs redirects" with
the community? in this discussion?
Look at this, and please tell me if it deserves to be shared:

  "Google images ➔ Mojeek image": {
    "regex": "^https?:\/\/([a-z]+\\.)*?google\\.[a-z]+\/search\\?(.*&)*(q=[^&]*).*",
    "replacement": "https:\/\/www.mojeek.com\/search?$3&fmt=images",
    "automatic": "false",
    "enabled": "true"
  },
  "Google search ➔ Mojeek search": {
    "regex": "^https?:\/\/(?:[a-z0-9-]+\\.)*?google.*\/(.*)",
    "replacement": "https:\/\/www.mojeek.com\/$1",
    "automatic": "false",
    "enabled": "true"
  },
  "Google images ➔ SearxNG image": {
    "regex": "^https?:\/\/([a-z]+\\.)*?google\\.[a-z]+\/search\\?(.*&)*(q=[^&]*).*",
    "replacement": [
      "https:\/\/searx.org\/search?$3&category_images=on",
      "https:\/\/metasearch.nl\/search?$3&category_images=on",
    ],
    "automatic": "false",
    "enabled": "true"
  },
  "Google search ➔ SearxNG search": {
    "regex": "^https?:\/\/(?:[a-z0-9-]+\\.)*?google.*\/(.*)",
    "replacement": [
      "https:\/\/searx.org\/$1",
      "https:\/\/metasearch.nl\/$1"
    ],
    "automatic": "false",
    "enabled": "true"
  },

Maybe just by adding them to the json files of @Raj9039852537, if he agrees with that :-)
And if you consider it useful to share it with community

[TrianguloY]

I think the best option is to upload it to a gist of your own, then link it in the wiki (do you have access to the wiki? If not tell me and I'll manage it).

You could also create a new discussion here and link that instead if you prefer.

I know it is a manual and tedious process, sorry, I need to think of a better way to manage them :/

[Ilithy]

No, no problem with that, I will create a json file just for the "anti-google" redirections, and another one, which is neither more nor less than the one of @Raj9039852537 but translated in French...
I will add this to the Wiki (yes I have write access to Wiki);

Thanks for your quick answer 🙏

You could also create a new discussion here and link that instead if you prefer.

Sorry, I didn't see that explained in the wiki 😅

[Raj9039852537]

Hello, @Ilithy
I have created a seprate repository and given you write access.

Feel free to upload your file "anti-google" there.

Also instead of gist this repository can be more beneficial since it can contain many confirutation files, with diffrent language setups also.

[Ilithy]

Hi @Raj9039852537 👍

[TrianguloY]

Nice! I'm not sure if I'll use that repository myself, and I personally prefer the wiki (it's here instead of in an external place outside my control) but it's definitely a nice addition, so I linked and marked it on the wiki.
If you think it's confusing or can be improved just say so!

[Ilithy]

Hi;
I have a question, is it impossible to add an exception to a Patterns checker rule?
Let me explain, I sometimes use TOR, and sites are all in http, so I wanted URLCheck to ignore ".onion" urls for the http/s rule, but it doesn't work, here it is:

  },
  "Http link, consider using https": {
    "regex": "^http:\/\/",
    "replacement": "https:\/\/",
    "exceptions": "http:\/\/([a-z0-9-])*?.onion\/(.*)",
    "enabled": "true"
  },

If you have ideas to make this work I'm interested. Otherwise, sorry to have bothered you.
Thank you <3

[Raj9039852537]

Hi
You have to paste it in regex string in place of

^http:\/\/

}, "Http link, consider using https": { "regex": "^http:\/\/(?!.*\\.onion\/?)(.*)", "replacement": "https:\/\/$1", "exceptions": "http:\/\/([a-z0-9-])*?.onion\/(.*)", "enabled": "true" },

That will do the job for now

Tested with http , onion and onion/ url

edit : corrected replacement string

[TrianguloY]

That's exactly what I was going to suggest, thanks Raj!
Regexp is extremely powerful, but also extremely difficult. I think that any exclusion functionality can already be done with it, but I agree it is simple enough to implement. Please create a new issue with it (it is easier to not forget it).

Just one think to consider raj, the regexp shouldn't have the leading (.*), otherwise the replacement will remove it! Basically the rule matches if the regexp matches anywhere in the url, and then every instance of the regexp is replaced by the replacement.

So, for example regexp "x" and replacement "" will remove all the 'x' letters from the url, but regexp "x(.*)" with replacement "" will remove everything after (and including) the first 'x'!

[Raj9039852537]

Yes definately and thanks

I forget to mention that in replace string, we need to use

https:\/\/$1

to mitigate this leading (.*) at the end

[TrianguloY]

Yes definately and thanks

I forget to mention that in replace string, we need to use

https:\/\/$1

to mitigate this leading (.*) at the end

True, but I think it's just easier to simply not adding the (.*) part, shouldn't be needed to change the replacement in that case.

[Ilithy]

  "regex": "^http:\/\/(?!.*\\.onion\/?)",
   "replacement": "https:\/\/",
   "exceptions": "http:\/\/([a-z0-9-])*?.onion\/",
   "enabled": "true",
   "automatic": "false"

I tested this model several times (given by @Raj9039852537 and modified by @TrianguloY to remove the (.*)) and it works as expected (thanks), URLCheck no longer reacts when it manages an address in .onion starting with http (the next step is that URLCheck reacts to an address in .onion in https to correct it, I will try a few things).

Thanks to you two very much for your help (and your patience)

[squeak9]

Easy way to check links with urlscan.io

  "Check with urlscan.io": {
    "regex1": "^(https?):\/\/(\\S*)$",
    "regex": "^(?!.*(urlscan.))(.*)",
    "replacement": "https:\/\/urlscan.io\/#$2",
    "enabled": "true"

EDIT: now that i look at this... will this work with http links? Maybe thats a thing missed by lots of these?
EDIT2: yes it works.
Maybe this should be its own topic..

[TrianguloY]

I think this is a copy of #260 (comment)

I'll add it to the wiki too, but with the same improvements as in #386 (comment) (and also mentioning the other author first)