URL Filters - SafeSearch
tim moore <tcruiser60-gM/[email protected]> Sun, 7 Dec 2014 08:26:59 +0000
| Newsgroups | gmane.comp.security.ipcop.devel |
|---|---|
| Message-ID | <CALLTnVpmGMgnJb7E0XOrkRD6qYHttReuka+B0DuFwT2x417OKQ@mail.gmail.com> |
--===============3064996425736090735== Content-Type: multipart/alternative; boundary=047d7b6773dcd7736305099c12d7 --047d7b6773dcd7736305099c12d7 Content-Type: text/plain; charset=UTF-8 ---------- Forwarded message ---------- From: *tim moore* <tcruiser60-gM/[email protected]> Date: Wednesday, December 3, 2014 Subject: URL Filters - SafeSearch To: [email protected] Hi First off apologies if this is the incorrect place to post this question, please direct me to the correct location if you have one. I am working on installing an IPCOP box in my daughters primary school to try and help them out. One of the key features is the Proxy and the filtering as these kids are between 4 and 9 years old. I have most features working OK but the issue I have run into is the Safesearch option for the URL filters. In the present state it just doesn't work. To be clear I am forcing the search over http by using the google site without SSL so this is not an issue of encrypted search. Searching the internet does give you any number of search and replace strings for this option, varying from complete rubbish to very complex. I have had limited success with a set of expressions I found over at pfsense web site (see below), with these the google searches work well enough possible not as expected. but trying to expand this to Bing just doen't work and I am not sure why. the search parameters are almost identical to google, their fore the replacement string should be the same. Now I realise that IPCOP is a home distribution and I am outside that spec trying to place it in a school. But if possible I would like to suggest a couple of small modifications. Firstly would it be possible to have the safesearch parameters in a separate file on the system rather than in the main urlfilter.cgi script ? I believe this would be useful to all, as the safe search options for each search engine change frequently and this would make updates/customization simpler and more effective. The second modification would be for Squid, in particular the cache directory location, this may be pushing my luck as it goes outside of the IPCOP design. The reason for this is to allow the addition of a dedicated disk for the cache. As far as I can tell the location for the cache directory is set in the proxy.cgi script, would it be possible to set this in side the settings file for the proxy. Under normal installations it would make no difference but it would allow for the installation to be easily modified. With both changes the aim will be to allow modifications without the changes to the main software and ensuing that updates still operate with out issue . Could some one confirm that we are on Squidguard 1.5Alpha, as this may be part of the issue ? I have posted notes in the user forum ( http://www.ipcops.com/phpbb3/viewtopic.php?f=3&t=18673), I am not that good at coding but think I could modifie and test the changes to the urlfilter.cgi script for the safesearch changes. And possible add the cache directory location parameter in proxy.cgi. But my abilities end their would someone be willing to submit the files for testing and possible integration into the code ? (The diff csv is a bit beyond my abilities for now) details below many thanks for reading - Tim if you go to line 3018 comment out to line 3029 then insert the following google works in a reasonable fashion. But I don't seem to be able to make bing work, not sure why. if ($filtersettings{'ENABLE_SAFESEARCH'} eq 'on') { print FILE "rewrite rew-rule-safesearch {\n"; print FILE " # rewrite safesearch\n"; print FILE " s@(google\\..*/search?.*q=.*)\@\\1\\&safe=active\@i\n"; print FILE " s@(google\\..*/images.*q=.*)\@\\1\\&safe=active\@i\n"; print FILE " s@(google\\..*/groups.*q=.*)\@\\1\\&safe=active\@i\n"; print FILE " s@(google\\..*/news.*q=.*)\@\\1\\&safe=active\@i\n"; print FILE " s@(yandex\\..*/yandsearch?.*text=.*)\@\\1\\&fyandex=1\@i\n"; print FILE " s@(search\\.yahoo\\..*/search.*p=.*)\@\\1\\&vm=r\@i\n"; print FILE " s@(search\\.live\\..*/.*q=.*)\@\\1\\&adlt=strict\@i\n"; print FILE " s@(search\\.msn\\..*/.*q=.*)\@\\1\\&adlt=strict\@i\n"; print FILE " s@(bing\\..*/search?q=.*/&)\@\\1\\&adlt=strict\@i\n"; print FILE " s@(bing\\..*/images/search?q=.*/&)\@\\1\\&adlt=strict\@i\n"; print FILE "}\n\n"; } This produces the following in the /var/ipcop/proxySquidGuard.conf file - rewrite rew-rule-safesearch { # rewrite safesearch s@(google\..*/search?.*q=.*)@\1\&safe=active@i s@(google\..*/images.*q=.*)@\1\&safe=active@i s@(google\..*/groups.*q=.*)@\1\&safe=active@i s@(google\..*/news.*q=.*)@\1\&safe=active@i s@(yandex\..*/yandsearch?.*text=.*)@\1\&fyandex=1@i s@(search\.yahoo\..*/search.*p=.*)@\1\&vm=r@i s@(search\.live\..*/.*q=.*)@\1\&adlt=strict@i s@(search\.msn\..*/.*q=.*)@\1\&adlt=strict@i s@(bing\..*/search?.*q=.*/&)@\1\&safe=strict@i s@(bing\..*/images/search?.*q=.*/&)@\1\&safe=strict@i } Sample URL's http://www.google.co.uk/?gfe_rd=cr&ei=k-N-VLC1FYjS8QPduoHoDQ&gws_rd=ssl#q=pictures http://www.bing.com/search?q=pictures&qs=n&form=QBLH&pq=pictures&sc=8-5&sp=-1&sk=&cvid=7c7f34ce68b74ce8adb3375fcfd82918 All tests are using a client with a dedicated proxy setting pointing to IPCOP on 8080 msn is now bing anyway and I have not looked at yahoo as this is still encrypted and I have to see if we can stop that. Apart from the actual insert I can't see why similar expressions do not work on bing !! any help please. I can see no indication in the log of any failure. --047d7b6773dcd7736305099c12d7 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable <br><br>---------- Forwarded message ----------<br>From: <b>tim moore</b> &= lt;<a href=3D"mailto:tcruiser60-gM/[email protected]">tcruiser60-gM/[email protected]</= a>><br>Date: Wednesday, December 3, 2014<br>Subject: URL Filters - SafeS= earch<br>To: <a href=3D"mailto:[email protected]">ipcop-dev= [email protected]</a><br><br><br><div dir=3D"ltr"><div><div><div><di= v><div><div><div><div><div><div><div><div><div><div><div><div>Hi <br><br></= div>First off apologies if this is the incorrect place to post this questio= n, please direct me to the correct location if you have one.<br><br></div>I= am working on installing an IPCOP box in my daughters primary school to tr= y and help them out. <br><br></div>One of the key features is the Proxy and= the filtering as these kids are between 4 and 9 years old. <br><br></div>I= have most features working OK but the issue I have run into is the Safesea= rch option for the URL filters. In the present state it just doesn't wo= rk. <br><br></div>To be clear I am forcing the search over http by using th= e google site without SSL so this is not an issue of encrypted search. <br>= <br></div>Searching the internet does give you any number of search and rep= lace strings for this option, varying from complete rubbish to very complex= . <br><br></div>I have had limited success with a set of expressions I foun= d over at pfsense web site (see below), with these the google searches work= well enough possible not as expected. but trying to expand this to Bing ju= st doen't work and I am not sure why. the search parameters are almost = identical to google, their fore the replacement string should be the same. = <br><br></div>Now I realise that IPCOP is a home distribution and I am outs= ide that spec trying to place it in a school. But if possible I would like = to suggest a couple of small modifications. <br><br></div>Firstly would it = be possible to have the safesearch parameters in a separate file on the sys= tem rather than in the main urlfilter.cgi script ? I believe this would be = useful to all, as the safe search options for each search engine change=C2= =A0 frequently and this would make updates/customization simpler and more e= ffective.=C2=A0 <br><br></div>The second modification would be for Squid, i= n particular the cache directory location, this may be pushing my luck as i= t goes outside of the IPCOP design.<br></div>The reason for this is to allo= w the addition of a dedicated disk for the cache. As far as I can tell the = location for the cache directory is set in the proxy.cgi script, would it b= e possible to set this in side the settings file for the proxy. Under norma= l installations it would make no difference but it would allow for the inst= allation to be easily modified. <br><br>With both changes the aim will be t= o allow modifications without=C2=A0 the changes to the main software and en= suing that updates still operate with out issue .<br><br></div><div>Could s= ome one confirm that we are on Squidguard 1.5Alpha, as this may be part of = the issue ?<br></div><div><br></div>I have posted notes in the user forum (= <a href=3D"http://www.ipcops.com/phpbb3/viewtopic.php?f=3D3&t=3D18673" = target=3D"_blank">http://www.ipcops.com/phpbb3/viewtopic.php?f=3D3&t=3D= 18673</a>), I am not that good at coding but think I could modifie and test= the changes to the urlfilter.cgi script for the safesearch changes. And po= ssible add the cache directory location parameter in proxy.cgi. But my abil= ities end their would someone be willing to submit the files for testing an= d possible integration into the code ? (The diff csv is a bit beyond my abi= lities for now)<br><br></div>details below many thanks for reading - Tim<br= ><br>if you go to line 3018 comment out to line 3029 then insert the follow= ing google works in a reasonable fashion.<br>But I don't seem to be abl= e to make bing work, not sure why.<br><br><br> if ($filtersettings{'= ENABLE_SAFESEARCH'} eq 'on') {<br> print FILE "rewr= ite rew-rule-safesearch {\n";<br> print FILE " # rewrit= e safesearch\n";<br> print FILE " s@(google\\..= */search?.*q=3D.*)\@\\1\\&safe=3Dactive\@i\n";<br> = print FILE " s@(google\\..*/images.*q=3D.*)\@\\1\\&safe=3Dactiv= e\@i\n";<br> print FILE " s@(google\\..*/groups= .*q=3D.*)\@\\1\\&safe=3Dactive\@i\n";<br> print FIL= E " s@(google\\..*/news.*q=3D.*)\@\\1\\&safe=3Dactive\@i\n"= ;;<br> print FILE " s@(yandex\\..*/yandsearch?.*text= =3D.*)\@\\1\\&fyandex=3D1\@i\n";<br> print FILE &qu= ot; s@(search\\.yahoo\\..*/search.*p=3D.*)\@\\1\\&vm=3Dr\@i\n";= <br> print FILE " s@(search\\.live\\..*/.*q=3D.*)\@\= \1\\&adlt=3Dstrict\@i\n";<br> print FILE " = s@(search\\.msn\\..*/.*q=3D.*)\@\\1\\&adlt=3Dstrict\@i\n";<br> = print FILE " s@(bing\\..*/search?q=3D.*/&)\@\\1\\&a= mp;adlt=3Dstrict\@i\n";<br> print FILE " s@(bin= g\\..*/images/search?q=3D.*/&)\@\\1\\&adlt=3Dstrict\@i\n";<br>= <br> print FILE "}\n\n";<br> }<br><br></div><div>This p= roduces the following in the /var/ipcop/proxySquidGuard.conf file - <br><br= ><br></div><div>rewrite rew-rule-safesearch {<br>=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 # rewrite safesearch<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 s@(google\..*/search?.*q=3D.*)@\1\&safe=3Dactive@i<br>=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s@(google\..*/images.*q=3D.*)@\1\&a= mp;safe=3Dactive@i<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s@(google\= ..*/groups.*q=3D.*)@\1\&safe=3Dactive@i<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 s@(google\..*/news.*q=3D.*)@\1\&safe=3Dactive@i<br>=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s@(yandex\..*/yandsearch?.*text=3D.= *)@\1\&fyandex=3D1@i<br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s@(s= earch\.yahoo\..*/search.*p=3D.*)@\1\&vm=3Dr@i<br>=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 s@(search\.live\..*/.*q=3D.*)@\1\&adlt=3Dstrict@i= <br>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 s@(search\.msn\..*/.*q=3D.*)= @\1\&adlt=3Dstrict@i<br>=C2=A0 =C2=A0 =C2=A0=C2=A0=C2=A0 s@(bing\..*/se= arch?.*q=3D.*/&)@\1\&safe=3Dstrict@i<br>=C2=A0 =C2=A0 =C2=A0=C2=A0= =C2=A0 s@(bing\..*/images/search?.*q=3D.*/&)@\1\&safe=3Dstrict@i<br= >}<br><br><br><br></div>Sample URL's <br><br><a href=3D"http://www.goog= le.co.uk/?gfe_rd=3Dcr&ei=3Dk-N-VLC1FYjS8QPduoHoDQ&gws_rd=3Dssl#q=3D= pictures" target=3D"_blank">http://www.google.co.uk/?gfe_rd=3Dcr&ei=3Dk= -N-VLC1FYjS8QPduoHoDQ&gws_rd=3Dssl#q=3Dpictures</a><br><a href=3D"http:= //www.bing.com/search?q=3Dpictures&qs=3Dn&form=3DQBLH&pq=3Dpict= ures&sc=3D8-5&sp=3D-1&sk=3D&cvid=3D7c7f34ce68b74ce8adb3375f= cfd82918" target=3D"_blank">http://www.bing.com/search?q=3Dpictures&qs= =3Dn&form=3DQBLH&pq=3Dpictures&sc=3D8-5&sp=3D-1&sk=3D&a= mp;cvid=3D7c7f34ce68b74ce8adb3375fcfd82918</a><br><br></div><div>All tests = are using a client with a dedicated proxy setting pointing to IPCOP on 8080= <br></div><div><br></div>msn is now bing anyway and I have not looked at ya= hoo as this is still encrypted and I have to see if we can stop that. <br>A= part from the actual insert I can't see why similar expressions do not = work on bing !! any help please.<br><br></div>I can see no indication in th= e log of any failure.<br><div><div><div><div><div><div><br></div></div></di= v></div></div></div></div> <br> --047d7b6773dcd7736305099c12d7-- --===============3064996425736090735== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------------ Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server from Actuate! Instantly Supercharge Your Business Reports and Dashboards with Interactivity, Sharing, Native Excel Exports, App Integration & more Get technology previously reserved for billion-dollar corporations, FREE http://pubads.g.doubleclick.net/gampad/clk?id=164703151&iu=/4140/ostg.clktrk --===============3064996425736090735== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ IPCop-devel mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/ipcop-devel --===============3064996425736090735==--