AVNXLog

This is our third session on getting Alienvault to play with NXLog. In the first two articles, we’ve explored on enabling TCP for rsyslog on Alienvault (thoroughly optional), and to forward NXLog logs (in this case FTP) to Alienvault. The problem is AV is having a hard time digesting the log and using the stock syslog as a plugin, is spewing out useless events with the IP as 0.0.0.0. The raw logs are fine.

We’re going to take a brief look at Plugins. I am not going to dive deep into plugins, there are plenty of source material out there from Alienvault and from Google, but a brief look here will do.

Plugins is the magic that interprets logs. If you head over to

/etc/ossim/agent/plugins

You fill find tons of plugins already written for you out of the box. Problem is there is bound to be something that isn’t supported. Now to be clear there are many ways to skin a cat, if you are into cat-skinning.

a) Let AV do the work. This is the best way. This way allows you to get down to doing your work, and unless your job description is actually sitting down writing Alienvault plugins, I would suggest this method. You need to have a maintenance contract with AV though.

https://www.alienvault.com/knowledge-base/how-to-request-a-new-plugin

b) Using the smart event collector.

https://www.alienvault.com/knowledge-base/smart-event-collection-how-to

Yes, there is actually a plugin creator out of the box! Again, you will need some basic config and you would think this is a Godsend GUI to solve all your plugins problem. Except – I believe it’s still in development on this as I fed it a couple of times and it gave some functional results, but struggled to customise to the one I wanted. It’s probably a good starting point, else, I suggest to roll up your sleeve and impress your co-workers by working manually on plugins. (after trying out route A, of course).

To summarise plugins, they consist of two files:

a) The CFG File – this is where all the stuff you need to write occurs. The suggestion is to just copy a current cfg file in the plugins directory and make it your own, and change the plugin_id.

b) The SQL file – this is found in /usr/share/doc/ossim-mysql/contrib/plugins. This is where once the cfg file is ready, we actually write the events we want to capture into the ossim-db.

Starting with the cfg file for the above example, here is a snapshot

# Alienvault plugin - CUSTOM
# Author: Professor Xavier
# Plugin iis_ftp id:10000 version: 0.0.1
# Last modification: 2016-04-01 13:00
#
# Plugin Selection Info:
# Microsoft:IIS Internet Information Services FTP:-
#
# END-HEADER

[DEFAULT]
plugin_id=10000

[config]
type=detector
enable=yes
source=log
location=/var/log/alienvault/devices/192.168.0.35/192.168.0.35.log
create_file=false
process=
start=no
stop=no
startup=
shutdown=

[translation]


[001 - FTP IIS - UNSUCCESSFUL LOGIN ]
regexp=(?P<date>\w{3}\s+\d{1,2}\s(\d{4}\s)?\d\d:\d\d:\d\d)\s(?P<device>\S+)\s\S*\s\S*\s\S*\s(?P<src_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(?P<src_port>\S*)\s(?P<username>\S*)\s\S*\s\S*\s\S*\s(?P<dst_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\s(?P<dst_port>\S*)\s(PASS)\s(?P<password>\S*)\s(530).*
event_type=event
date={normalize_date($date)}
plugin_sid=1
device={$device}
src_ip={$src_ip}
src_port={$src_port}
username={$username}
dst_ip={$dst_ip}
dst_port={$dst_port}
userdata1={$password}

There are a few key components here. First, the header is a must. Second, this is a simplified example whereby I did not use any translation yet. What these ‘translations’ are, are simply a way to interpret events that look the same and change these so-called identifiers to numeric plugin_sid so you don’t end up writing extra Regex. It sounds important, but for a demo, let’s leave it empty for now and we will see that you actually don’t need it to run. It’s like a spoiler on a car. It makes your file looks good, but your car can run fine.

By the way Regex is regular expression, and its basically in python format.

The headers are necessary here, and after that are some default values that are self explanatory. The work starts at the bottom. You generally need to have

a) A Specific rule (that refers to a specific event – in this case UNSUCCESSFUL LOGINS)

b) A catchall rule (which I did not write here, due to time), but it’s generally where all the other events are caught, usually with the help of translation tables.

In this example, we have Unsuccessful Login.

The key is to take the actual unsuccessful login log from the raw log file itself, presented:

Apr  3 16:48:05 S006-SVR01 IIS[1] 2016-04-03 08:46:51 192.168.0.1 5490 - FTPSVC2 S006-SVR01 - 192.168.0.35 21 PASS *** 530 1326 41 25 17 0 6080e2d2-0565-4eb1-9eec-407683eac92a -#015

And head over to https://regex101.com/, and do some testing.

Make sure you select your ‘Flavor’ on the left menu as PYTHON, and then put in the raw log line under ‘TEST STRING’. Now you need to start doing the regular expression above, under “REGULAR EXPRESSION” (WOW!).

The thing about Regex is, it’s like programming. There is a way to do it that it works, and there is a way to do it that it works beautifully. I am the type of pseudo programmer whereby I would always forget to comment or to forget to do memory management on my C code in university. Instead of building functions and classes etc I would just create long lines of inefficient code just to get the work done and receive a barely passing mark and a long look of disapproval from my university lecturers. In this case, efficient code is key because if you are running tons of lines of logs through AV, the last thing you want your box to do is to wade through a pile of inefficient REGEX to understand it.

Thankfully, this article isn’t about efficient regex, so I suggest to really get cracking on it, or get Alienvault regex gurus to work on it. Instead, what you see above is a very simple regex just to show some demo that AV can actually grab logs, interpret it and put it into an event in the SIEM. The regex above is rife with /s and /S, basically just saying, there is a whitespace, or there is a non-whitespace (characters).

However, you will notice that we assign a few variables as well – namely, IP addresses, usernames , device, ports and of course date.

normalize_date is an inbuilt function to ensure that the date in the raw log is normalized to something that AV understands.

The plugin_sid is also important, as it assigns the event (within this plugin) a place in the database and allows AV to work on it and identify it. AV also allows 9 userdata slots to be used for other reasons for identification. So in this case, we want to match whatever we see in the logs that has ‘PASS’ and the errorcode ‘530’ in it and we would know this means that someone tried to login unsuccessfully.

Once the demo cfg is ready, you need to build your SQL File.

The SQL file, again, you can just take it from another SQL file in the SQL folder, and rename it appropriately. It’s a lot shorter and it might go something like this:

DELETE FROM plugin WHERE id = "10000";
DELETE FROM plugin_sid where plugin_id = "10000";
INSERT IGNORE INTO plugin (id, type, name, description) VALUES (10000, 1, 'IIS_FTP', 'IIS FTP');
INSERT IGNORE INTO plugin_sid (plugin_id, sid, category_id, class_id, name, priority, reliability) VALUES (10000, 1, NULL, NULL, 'UNSUCCESSFUL LOGIN', 4, 3);

That’s pretty much it. First, you delete the plugin and then insert a new one with the required descriptions and names and put in the plugin_sid (Event Type IDs), in this case “1”, which equates to UNSUCESSFUL LOGIN.

Run the command

cat iis_ftp.sql | ossim-db

to write the sql to the DB (of course using the actual SQL name you have).

And you are now ready to do some testing!