Re: Dynamic hosts (eg no-ip) and fail2ban

Sorry, the ipset should be called: whitelistednames

On Wed, Dec 23, 2015 at 9:01 PM, Tsaousis, Costa <[email protected]> wrote:
> yes, it works.
>
> If you now use an ipset called 'whitelist' in your firewall, update
> ipsets will update it when the contents of the resulting
> whitelist.ipset change.
>
> FireHOL v3 depends on iprange but they are separate packages.
> There is no need to do anything. The iprange fix is already released.
>
>
> On Wed, Dec 23, 2015 at 8:52 PM, Jason Harris <[email protected]> wrote:
>> It didn’t work with the version I compiled using the sources of around 10 to
>> 15 days ago.
>>
>> root@testfirehol:~# printf
>> "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
>> root@testfirehol:~# man iprange
>> No manual entry for iprange
>> See 'man 7 undocumented' for help when manual pages are not available.
>> root@testfirehol:~# iprange --version
>> 1.0.2_master
>> root@testfirehol:~#
>>
>> I’ll try again now.
>>
>> Ahh… That new change fixed the single test you gave above. After rebuilding
>> the droplet and reinstalling from git sources I get the following:
>>
>> root@testfirehol:~# printf
>> "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
>> 5.196.125.115
>> 23.235.43.133
>> 173.194.65.99
>> 173.194.65.103
>> 173.194.65.104/31
>> 173.194.65.106
>> 173.194.65.147
>> root@testfirehol:~# iprange --version
>> 1.0.3_master
>>
>> One question the fix that you put in, did it make it into 3.0.0?
>>
>> In any case the instructions now work as you described them. I get the
>> following:
>>
>> ...
>>                   whitelistednames| parsing attributes:
>>                                    | source file has been updated
>>                                    | converting with 'hostname_resolver'
>> iprange: DNS: 'bob.robsfdsd.com' failed permanently: Name or service not
>> known
>>                                    | parsing attributes:
>>                                    |  SAVED  no need to load ipset in kernel
>>                                    | version 3, 1 unique IPs
>> '/etc/firehol/ipsets.d/whitelistednames.conf' completed
>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist.
>> Ignoring it.
>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist.
>> Ignoring it.
>>
>> Cleaning up temporary files in /tmp/update-ipsets-BX6iJZO6d6.
>> Completed successfully.
>>
>> root@testfirehol:/etc/firehol/ipsets# more whitelistednames.source
>> bob.robsfdsd.com
>> namecheap.com
>>
>> root@testfirehol:/etc/firehol/ipsets# more whitelistednames.ipset
>> #...
>> 199.59.161.100
>> root@testfirehol:/etc/firehol/ipsets#
>>
>> So this looks like it is really working correctly!
>>
>> Thank you so much!
>>
>> Cheers,
>>    Jason
>>
>> On Dec 23, 2015, at 10:35 AM, Tsaousis, Costa <[email protected]> wrote:
>>
>> I thought you solved it.
>>
>> I see it says: converting with 'hostname_resolver'
>> But then: ERROR  converted file is empty.
>>
>> The key question is if hostname_resolver works for your list.
>>
>> What hostname_resolver does is essentially this:
>>
>> iprange </etc/firehol/ipsets/mywhitelist.source
>>
>> /etc/firehol/ipsets/mywhitelist.ipset
>>
>>
>> Does it work if you run it by hand?
>>
>> For example:
>>
>> printf "www.google.com\nfirehol.org\niplists.firehol.org\n" | iprange
>>
>> Keep in mind I found a bug, in iprange where the last few hostnames in
>> the source set, in certain cases were resolved, but were not included
>> in the result. Try the github version of iprange.
>>
>> Costa
>>
>>
>> On Wed, Dec 23, 2015 at 7:31 AM, Jason Harris <[email protected]>
>> wrote:
>>
>> Hey Tsaousis,
>>
>> Did you get a chance to see what was wrong with the configuration, below? I
>> am still really keen to get something going here. (I really like firehol and
>> have just been using it for some of the networking on some LXC containers.
>> It really allows the vey nice specification of firewall / routing rules.)
>>
>> Along with Phil’s comments I can now create .deb’s for firewall and deploy
>> them so everything is set up except for these ipsets…
>>
>> Thanks for any help!
>>   Jas
>>
>> I just spun up a new instance as an example on here. There is nothing else
>> but a stock install of debian 8.2 hosted on digital ocean.
>>
>> I apt-get installed: git sudo python-pip autoconf build-essential curl ipset
>> Then ran the script at:
>> https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets
>>
>> I’ll send you the login detail in a separate email. But here is the output
>> of `update-ipsets -v` with the contents of the files following:
>>
>>                                  |
>>                 firehol_anonymous|  DISABLED
>>                                  | To enable run: update-ipsets enable
>> firehol_anonymous
>> Loading ipset definitions from: '/etc/firehol/ipsets.d'
>> Loading ipset definition file: '/etc/firehol/ipsets.d/mywhitelist.conf'
>>                                  |
>>                       mywhitelist| parsing attributes:
>>                                  | converting with 'hostname_resolver'
>>                                  |  ERROR  converted file is empty.
>> ERROR : '/etc/firehol/ipsets.d/mywhitelist.conf' failed
>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist.
>> Ignoring it.
>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist.
>> Ignoring it.
>>
>> Cleaning up temporary files in /tmp/update-ipsets-O3Z0xZFOWe.
>> Completed successfully.
>> root@testfirehol:/etc/firehol/ipsets# cd ..
>> root@testfirehol:/etc/firehol# ls
>> firehol.conf.example  fireqos.conf.example  ipsets  ipsets.d  services
>> root@testfirehol:/etc/firehol# more ipsets/*
>>
>> *** ipsets/errors: directory ***
>>
>>
>> *** ipsets/history: directory ***
>>
>> ::::::::::::::
>> ipsets/mywhitelist.source
>> ::::::::::::::
>> google.com
>> yahoo.com
>> cnn.com
>> namecheap.com
>> root@testfirehol:/etc/firehol# more ipsets.d/*
>> # update its timestamp, to force reprocessing
>> touch /etc/firehol/ipsets/mywhitelist.source
>>
>> # configuration about the list
>> update mywhitelist 1 0 ipv4 ip "" hostname_resolver "category" "a whitelist
>> for me" "Jason Harris" "a url for info for the list"
>> root@testfirehol:/etc/firehol#
>>
>> Thanks!
>>  Jason
>>
>>
>> On Dec 13, 2015, at 5:19 AM, Tsaousis, Costa <[email protected]> wrote:
>>
>> to disable a list in update-ipsets, just delete its .source file.
>> The enable command just touches it.
>>
>> Regarding the conversion error, could you please post
>> /etc/firehol/ipsets/whitelist.source?
>>
>> If you don't want to post your whitelist, this should work:
>>
>> iprange </etc/firehol/ipsets/whitelist.source
>>
>> it should give you the IPs of your hostnames.
>>
>> Costa
>>
>> On Sat, Dec 12, 2015 at 8:02 PM, Jason Harris <[email protected]>
>> wrote:
>>
>>
>> On Dec 8, 2015, at 1:48 PM, Tsaousis, Costa <[email protected]> wrote:
>>
>> Ok. but I can use hostnames like eg sub.mydomain.com with ipsets?
>>
>>
>> Yes, you have to resolve them first though. iprange does this.
>>
>>
>> The link:
>> https://github.com/ktsaou/firehol/blob/master/contrib/update-ipsets.sh on
>> the page: https://github.com/firehol/firehol/wiki/Working-with-IPSETs is
>> dead. I google around a bit and am sure I am just missing this but am having
>> trouble finding this script.
>>
>>
>> Thanks! I fixed the link.
>> However, it is installed with firehol v3 (the github version).
>>
>>
>> So I am not sure how to actually update the ipset I have dynamically. Maybe
>> I could build a second ipset and using 'ipset swap’? But it seems to be from
>> the instructions below that I should use update-upsets?
>>
>>
>> ok.
>>
>> 1. Install firehol v3 (this will also require from you to install
>> iprange). If you don't know how to do it, follow this procedure:
>> https://github.com/firehol/blocklist-ipsets/wiki/Installing-update-ipsets
>>
>>
>> Ok. I got around to having some time this weekend. To build this (on latest
>> debian jessie) in addition to your listed build steps you also need:
>>
>> apt-get install autoconf build-essential curl ipset
>>
>> This is kind of disappointing since it loads a bunch of gunk onto a
>> production node, (i.e. some 200MB’s of stuff just to get the small firehol
>> firewall. I guess I could remove most of this after the build process… Still
>> this is not so nice for eg ansible,chef, puppet, saltstack, etc which are
>> used to provision vm’s.)
>>
>> 2. Create a new file called /etc/firehol/ipsets/myhostsnames.source
>> Put there any hostnames you like.
>>
>> 3. To resolve its contents to IPs you have to configure update-ipsets
>> (https://github.com/firehol/blocklist-ipsets/wiki/Extending-update-ipsets).
>> Briefly:
>>
>> a. create the file  /etc/firehol/ipsets.d/myhostname.conf
>> b. using this content (copy and paste it):
>>
>> # update its timestamp, to force reprocessing
>> touch /etc/firehol/ipsets/myhostsnames.source
>>
>> # configuration about the list
>> update myhostnames 1 0 ipv4 ip "" hostname_resolver "category" "some
>> info about the list" "your name" "a url for info for the list"
>>
>> c. run:
>>
>> update-ipsets enable myhostnames
>>
>>
>> Ok. So I followed these instructions. First there appears to be no
>> update-ipsets disable myhostnames? (I made a mistake in one of the
>> configurations and it would be nice to undo it…)
>>
>> d. check it with (this is also the command you need to put at cron):
>>
>> update-upsets
>>
>>
>> For me this fails with the following message (using update-upsets -v)
>>
>>                firehol_anonymous|  DISABLED
>>                                 | To enable run: update-ipsets enable
>> firehol_anonymous
>> Loading ipset definitions from: '/etc/firehol/ipsets.d'
>> Loading ipset definition file: '/etc/firehol/ipsets.d/whitelist.conf'
>>                                 |
>>                        whitelist| parsing attributes:
>>                                 | converting with 'hostname_resolver'
>>                                 |  ERROR  converted file is empty.
>> ERROR : '/etc/firehol/ipsets.d/whitelist.conf' failed
>> Supplied ipsets directory '/usr/share/firehol/ipsets.d' does not exist.
>> Ignoring it.
>> Supplied ipsets directory '/root/.update-ipsets/ipsets.d' does not exist.
>> Ignoring it.
>>
>> Cleaning up temporary files in /tmp/update-ipsets-9B34pYTy0N.
>> Completed successfully.
>> [root@tester:/etc/firehol/ipsets] $ ls
>>
>> Any hints on what went wrong? The errors directory is empty...
>>
>> Thanks!
>> Jason
>>
>> If successful, the file /etc/firehol/ipsets/myhostnames.ipset should
>> be there with all the IPs.
>>
>> 4. In firehol.conf use
>>
>> ipset4 MYHOSTNAMES addfile ipsets/myhostnames.ipset
>>
>> and later in server/client/nat statements: src ipset:MYHOSTNAMES
>>
>>
>>
>>
>>
>>
_______________________________________________
Firehol-support mailing list
[email protected]
http://lists.firehol.org/mailman/listinfo/firehol-support