PKF AvantEdge

Category: PCI-DSS (Page 5 of 20)

Alienvault: Working with Decoders and Rules

March 29, 2021 / / 0 Comments

When we started out with Alienvault years ago, they were just a smallish, start up company and we worked directly almost with the engineers and sales team in Cork. Of course, a lot has changed since AT&T took over, but during the early days, there were a lot of knowledge and mindshare done directly between us and them. So much so that if you were to check their partner site, they still list us as the only Malaysian company as their reseller, due to the early days of listing. What attracted us to the product was that we could lift the hood and see what was underneath. Alienvault (or OSSIM) was previously a hodgepodge of many working parts that were glued together and somehow made to work. The agent was a product called OSSEC, which is an open-source HIDS. The IDS is Suricata/Snort and if you look closely at the availability tool, you would see the backend is a Nagios running. NFSen is used for their netflow data display, and PRADS for their asset discovery. OPENVAS is their vulnerability scanner and best of all, they allow you to jailbreak the system and go into the OS itself and do what you need to do. In fact, most of the time, we are more comfortable on the command line than through the actual UI itself.

The history aside, the downside of adding in these different applications and getting them all to play nice together, is that you would have to understand the interworkings of these pieces.

For instance, if you were to send logs via Syslog to Alienvault, you would have to know that the daemon rsyslog (not an Alienvault product) is the one being used to receive these logs. If you were to use the agent, then the application receiving these logs is different – it’s the OSSEC server that receives it. So it depends how logs come in, and from there you can decide what you wish to do with it.

The challenge is oftentimes to filter and ‘massage’ the logs when it hits Alienvault. There are a few approaches to this:

The basics are at stage 1 where the client (server, workstation etc) send logs (or have logs to be collected) to Alienvault. The initial filtering should theoretically happen here if possible. Many applications have the capability to control their logs – Windows server being one of them. Turning on debug logs on Linux for instance would cause a fair bit of log traffic across the network. Applications as well, have options of what to log and what not to log. We see firewalls logging traffic logs, proxies logging every single connection that goes through – this causes loads of logs hitting the Alienvault.

AV (especially the All In Ones) isn’t designed to take on heavy loads the way Splunk or other enterprise SIEM like ArcSight, that chews through 100,000 EPS like Galactus chews through planets. The AV approach has always been, we aren’t a SIEM only, we are a unified security management system, so security logs are what we are after. Correlation is what we are after. APT are what we are after. Their philosophy isn’t to overload and do generic Business Intelligence with millions of log lines, but to focus on Security and what is happening to your network. That being said, it’s no pushover as well, being able to work with 90 – 120 million events and going through 15,000 EPS on their enterprise.

The reality however is that most clients just turn on logs at Item 1 and plow these logs over to Alienvault. So it’s really up to Alienvault to start filtering these logs and stopping them coming in. At layer 2, is what we call the outer layer. This is the front line defence against these attacks of logs. These are where the engine running these log systems (OSSEC, rsyslog etc) can filter out and then trickle what is needed to Alienvault main engine itself in Layer 3. The AV main engine also has its form of defence, in policies, where we can create ‘junk’ policies to simply ignore logs coming in and not process them through the resource intensive risk assessment calculations.

So, we are going to assume that Layer 1 filtering wasn’t done. What we are going to look at is sorting out Layer 2 and we will assume that logs are coming in via OSSEC. We will have another article on Rsyslog filtering because that is a whole different novel to write.

When it hits OSSEC, it’s going via default port 1514/udp. Now remember, when logs first enters Alienvault, it doesn’t immediately go into the SIEM event display. It first needs to be logged, before it can be turned into events, before it can trigger alarms. So the basic rule is to get it logged:

Make sure you are receiving logs first.

This may seem juvenile in terms of understanding but we have been through enough to know that no matter WHAT the client says, oftentimes, their systems are not even sending the logs to us! A simple tcpdump -Xni eth0 “udp port 1514” will see if the logs are getting in, so go ahead with that first to ensure you are receiving. Just add a “and host <ip address>” if you need to filter it by the IP address.

Another way that Alienvault allows, when you are getting logs via HIDS/OSSEC is by enabling the “logall” on USM HIDS configuration, which we covered in the previous articles here. But be aware turning on logall potentially will bring a lot of logs and information into the box so we generally avoid this unless it’s really needed.

Once you are seeing logs coming into Alienvault, for OSSEC at least the next thing to do is to move these logs to “alerts.log” and from there, Alienvault can start putting it into the SIEM display.

For this to happen, you need to understand 3 things here, aside from the fact that we are currently now working on layer 2 from the diagram above – OSSEC:

a) Decoders

b) Rules

c) /var/ossec/bin/ossec-logtest

The above are actually OSSEC terminologies – not strictly Alienvault. What this means is that if you were to decouple OSSEC from Alienvault, you can. You can just download OSSEC. Or you could download other products like Wazuh, which is also another product we carry. Wazuh runs OSSEC (its own flavor) but has a different presentation layer (Layer 3 in our diagram above) and integrates with ELK to provide a more enterprise ready product but the foundation came from the same OSSEC principles. So when we talk about Rules and Decoders and using the ossec-logtest script to test your stuff, it’s not an Alienvault specific talk. Alienvault specific talk we can go later with plugins and stuff. In the actual ACSE course from Alienvault (at least the one I passed 5 years ago), there is really no mention on decoders and rules – it basically just focus on the core Alienvault items only.

At this point, we need to make the decision on whether to have the filtering done on OSSEC level (2) or on Alienvault level (3)? As a rule, the closer the filtering is done to source, the better…however, in our opinion, the filtering by Alienvault plugins is a lot more flexible and intuitive in design, compared to OSSEC (and because we are biasedly trained in Alienvault, but not so much in OSSEC). So for this article (which is taking VERY long in getting to its point), we are tasked to simply funnel the logs into /var/ossec/logs/alerts/alerts.log because that is where OSSEC sends its logs to and where we can get our AV plugins to read from.

The logs in /var/ossec/logs/archives/archives.log (remember, we turned on the logall option in the OSSEC configuration for this illustration) aren’t monitored by plugins. Because in a production environment, you won’t have that turned on. So, once you have logs into the alerts.log file, you are good to go, because then you can sit down and write plugins for Alienvault to use in the SIEM display.

OK – Firstly Decoders. OSSEC has a bunch of default decoders (like plugins in Alienvault) that is able to interpret a whole bunch of logs coming in. Basically, the decoder is set up with Regular expression to go through a particular file and just grab the information from the file and drop it into fields like IP address, date, source IPs etc. Similar to the AV plugin, but for this illustration, we are not going to use much of the OSSEC filtering, but simply to ensure we select the right logs and send them over to the alerts.log file.

So ok, let’s take the previous article example of having MySQL logs into Alienvault. Let’s say we have this example query log coming into our Alienvault (archive.log, if we turned it on)

2021 Feb 21 00:46:05 (Host-192-168-1-62) 192.168.1.62->\MySQLLOG/db.log 2021-02-22T09:41:42.271529Z        28 Query     SHOW CREATE TABLE db.persons

So the above doesn’t really offer much, but you can technically see there is the date and time, and the command line etc and a decoder will need to be created to parse the incoming log.

Picking up from where we left off at the Alienvault link, Task 4 covers the steps to create the decoder:

a) Edit /var/ossec/alienvault/decoders/local_decoder.xml and add in the following:

<decoder name="mysql-query">
        <prematch> Query</prematch>
</decoder>
<decoder name="mysql-connect">
        <prematch> Connect\s*</prematch>
</decoder>
<decoder name="mysql-quit">
        <prematch> Quit</prematch>
</decoder>

The above is simplistic decoder to catch the 3 important events from the logs coming in from MySQL – Query log, i.e

2021-02-22T09:41:42.271529Z        28 Query     SHOW CREATE TABLE db.persons

Connect Log

2021-02-20T16:35:28.019734Z        8 Connect   root@localhost on  using SSL/TLS

Quit

2021-02-20T18:29:35.626687Z       13 Quit

Now of course, for those aware, the Query logs have many different types of query – Query Use, Query Show, Query Select, Query Set, Query Insert, Query Update and so on. The idea of the decoder is simply to catch all the queries, and we will theoretically log all Queries into Alienvault.

Now, remember to tell Alienvault you have a new decoder file

In the USM Appliance web UI, go to Environment > Detection > HIDS > Config > Configuration.

Add <decoder>alienvault/decoders/local_decoder.xml</decoder> after <decoder> :

Adding the "local_decoder.xmll" setting to ossec_config

Adding this setting enables the usage of a custom decoder. Save it and restart HIDS.

So that’s it for the decoder.

Now, on the CLI, go to /var/ossec/bin and run ./ossec-logtest

Paste the following “2021-02-20T18:29:43.189931Z 15 Query SET NAMES utf8mb4”

And you should the get result as below

linux:/var/ossec/bin# ./ossec-logtest
 2021/03/29 09:50:10 ossec-testrule: INFO: Reading decoder file alienvault/decoders/decoder.xml.
 2021/03/29 09:50:10 ossec-testrule: INFO: Reading decoder file alienvault/decoders/local_decoder.xml.
 2021/03/29 09:50:10 ossec-testrule: INFO: Started (pid: 25070).
 ossec-testrule: Type one log per line.
 2021-02-20T18:29:43.189931Z        15 Query     SET NAMES utf8mb4
 **Phase 1: Completed pre-decoding.
        full event: '2021-02-20T18:29:43.189931Z    15 Query     SET NAMES utf8mb4'
        hostname: 'linux'
        program_name: '(null)'
        log: '2021-02-20T18:29:43.189931Z           15 Query     SET NAMES utf8mb4'
 **Phase 2: Completed decoding.
        decoder: 'mysql-query'

So basically, any logs that come into archive.log that has that sample line “Query” you will be lumping it in as mysql-query decoded. Of course you can further refine it with Regular expression to get the exact term you wish, but for the illustration, we want to catch the queries here and it’s fine for now.

The next item is the rules. Again, referring to the Alienvault writeup above, go ahead and edit
/var/ossec/alienvault/rules/local_rules.xml.

What we will do is to add the following in

<group name="mysql-connect">
<rule id="192000" level="0">
<decoded_as>mysql-connect</decoded_as>
<description>Connect log is enabled</description>
</rule>

<rule id="192001" level="1">
<if_sid>192000</if_sid>
<regex>Connect\s*</regex>
<description>Connection is found</description>
</rule>
</group>


<group name="mysql-query">
<rule id="195000" level="0">
<decoded_as>mysql-query</decoded_as>
<description>Mysql Query log is enabled!</description>
</rule>


<rule id="195001" level="0">
<if_sid>195000</if_sid>
<match>SET</match>
<description>Query set is found and ignored!</description>
</rule>


<rule id="195002" level="1">
<if_sid>195000</if_sid>
<regex>Query\s*</regex>
<description>Query is found</description>
</rule>
</group>


<group name="mysql-quit">
<rule id="194000" level="0">
<decoded_as>mysql-quit</decoded_as>
<description> Quit log is enabled</description>
</rule>

<rule id="194001" level="1">
<if_sid>194000</if_sid>
<regex>Quit\s*</regex>
<description>Quit command is found</description>
</rule>
</group>

So what the above does is to decide what to do with 3 types of MySQL logs you are getting: Connect, Query and Quit. We want to dump these logs into alerts.log so that we can work on it with Alienvault’s plugin. We don’t want to do any fancy stuff here so it’s pretty straightforward.

Each of these 3 have a foundation rule

a) Connect – 192000

b) Quit – 194000

c) Query – 195000

Each rule has a nested rule to decide what to do with it. Notice you can actually do Regex or Match on the rules which really provides a lot of flexibility in filtering. In fact, if it wasn’t for Alienvault’s plugins, OSSEC’s filtering would probably be sufficient for most of your custom logs requirement.

For this illustration, our job is simple – for each of these rules, find out the key word in the log, and then escalate it to an alert. An alert is created when you create a rule ID with level = 1, i.e <rule id=”195002″ level=”1″>

If you run ossec-logtest again, and paste the log there, you would be able to see

**Phase 1: Completed pre-decoding.
        full event: '2021 Feb 21 00:46:46 (Host-192-168-1-62) 192.168.1.62->\MySQLLOG/db.log 2021-02-22T09:42:21.711131Z        28 Quit'
        hostname: '(Host-192-168-1-62)'
        program_name: '(null)'
        log: '192.168.1.62->\MySQLLOG/db.log 2021-02-22T09:42:21.711131Z        28 Quit'
 **Phase 2: Completed decoding.
        decoder: 'mysql-quit'
 **Phase 3: Completed filtering (rules).
        Rule id: '194001'
        Level: '1'
        Description: 'Quit command is found'
 **Alert to be generated.

Once you see “alert to be generated” you will find that same alert in the /var/ossec/logs/alerts/alerts.log

AV - Alert - "1613881201" --> RID: "197011"; RL: "1"; RG: "connect"; RC: "Quit Command found"; USER: "None"; SRCIP: "None"; HOSTNAME: "(Host-192-168-1-62) 192.168.1.62->\MySQLLOG/db.log"; LOCATION: "(Host-192-168-1-62) 192.168.1.62->\MySQLLOG/db.log"; EVENT: "[INIT] 2021-02-22T09:42:21.711131Z        28 Quit       [END]";

From there, you can go about doing the plugins and getting it into the SIEM.

Whew. That’s it.

You would notice, however, there is another sub-rules in there for Query:

<rule id="195001" level="0">
<if_sid>195000</if_sid>
<match>SET</match>
<description>Query set is found and ignored!</description>
</rule>

This is set above the “alert” rule and you notice that this is Level=0. This means whatever Query that is decoded, first runs this rule and basically if I see there is a Query “SET”, I am going to ignore it. I.e it’s not a log I want and I am not going to put it into the alerts.log. Level 0 means, not to alert.

I am ignoring Query Set because in this case, we are finding millions of query set as it is invoked a lot of times and mostly it is false positives. I am interested in Query Selects, Inserts and Updates etc.

Once you have this rule put in, it will filter out all Query Sets. This is basically the only filtering we are doing so we don’t have those millions of Query Sets jamming up my alerts.log file in Alienvault.

alienvault:/var/ossec/logs/archives# ossec-logtest
 2021/03/14 12:36:33 ossec-testrule: INFO: Reading decoder file alienvault/decoders/decoder.xml.
 2021/03/14 12:36:33 ossec-testrule: INFO: Reading decoder file alienvault/decoders/local_decoder.xml.
 2021/03/14 12:36:33 ossec-testrule: INFO: Started (pid: 12550).
 ossec-testrule: Type one log per line.
 192.168.1.62->\MySQLLOG/db.log 2021-03-14T16:22:58.573134Z        19 Query     SET NAMES utf8mb4'
 **Phase 1: Completed pre-decoding.
        full event: '192.168.1.62->\MySQLLOG/db.log 2021-03-14T16:22:58.573134Z        19 Query     SET NAMES utf8mb4''
        hostname: 'alienvault'
        program_name: '(null)'
        log: '192.168.1.62->\MySQLLOG/db.log 2021-03-14T16:22:58.573134Z        19 Query     SET NAMES utf8mb4''
 **Phase 2: Completed decoding.
        decoder: 'mysql-query'
 **Phase 3: Completed filtering (rules).
        Rule id: '195001'
        Level: '0'
        Description: 'Query set is found and ignored!'

So you see, from the above, all Query Sets are ignored. You can basically do whatever you wish by using either Regex or Match and ignore certain log messages from OSSEC itself. It’s very powerful and flexible and with enough time and effort, you can really filter out only the needed logs you want into Alienvault, which is really part of the fine-tuning process for SIEM.

So there you have it. What you have done now is to take those logs from archives.log and make sure you only put the logs you want in alerts.log (Quit, Connect, All Query except for Query Set).

The next thing you need to do is to go down to Alienvault (layer 3) and do the heavy lifting in writing plugins and get these events into the SIEM display.

For more information for Alienvault and how it can help your compliance, send us an email at alienvault@pkfmalaysia.com and we will get back to you ASAP!

PCI-DSS: Estimating the Cost

February 7, 2021 / / 0 Comments

Ah money.

This is how most conversations start when we receive calls from PCI. How much will it cost?

I think this is one of the toughest subject for PCI, because it really depends on what is being done by the service provider/consultant for you, and how much you can actually do the implementation of PCI-DSS on your own. And obviously it also depends on your scope, and on top of that, depends on compensating controls if any, or any current controls you have in place. And then it also depends on the validation type – SAQ vs RoC and so on.

So, in the classic riposte to this classic question, it would be “It depends”.

Where we really need to clear the air though is the myth that once you have done PCI-DSS the first time, everything gets easier on the renewals and everything gets cheaper year on year going forward. That is for another article. There is a lot of things going on in PCI-DSS, and if you approach it from a product perspective (like most procurement do), you end up either sabotaging your entire compliance, or getting an auditor willing to sign off on God knows what, and later on realise that you’ve been out of compliance scope all the while.

To start with the pricing, you should understand a bit on the cost of PCI-DSS. And we should start with the QSA, because after all they are the focal point of the PCI program. They are the Qualified Security Assessor. Of course, you can opt to do your PCI (if allowed) without a QSA involvement (Merchant level 3 or 4) and just fill up an SAQ with or without assistance from consultants; but for the most part, a QSA would be involved in the signoff for larger projects, and this is where the cost questions take life.

Lets look firstly at the base cost of becoming a QSA. It’s very helpfully listed for us here: https://www.pcisecuritystandards.org/program_training_and_qualification/fees

So here are the maths. Imagine you are a QSA with projects in Malaysia: to start off, you will need to set aside over RM100K just to get you qualified to to audits in the Asian Region. We’re not talking about Europe or Latin America or USA here. Just APAC. That’s qualifying the company. A company, to service any region properly will probably need a bunch of QSAs trained and ready, let’s say around 3 to start off with. Each QSA will need to go for a training costing around RM12 – 13K, so let’s say you have 3 (which is very few), you are setting aside around MYR 50K for that. On top of that, there are obligations such as Insurance Coverage that is specified in the QSA Qualifications Requirement document. So it depends on which insurance you are taking, but it could be in the region of around MYR6K or above premium (spitballing). There is a requalification each year as well.

QSAs then can make their own calculations on how fast/long they need to recover their cost, but let’s say they set aside 200K just to get things set up with 3 or 4 QSAs, then they need to recover that cost. A man day of a QSA/Consultant may range from quite widely in this region but let’s say you decide to price it at “meagre” MYR2K, depending on how senior you have, so overall, you would need to have almost around 1.5 months of engagement of their QSAs just to recover the cost of setting up shop. That’s why its not unreasonable to see higher rates, because of the cost it takes.

You have salaries to consider as well. You also have to consider if something happens to one of your clients, where you happily audited them remotely and believed everything they said, and found out that they have done jack-shoot in their actual environment and you have to handle the fallout of liabilities.

Some procurement compares QSA engagements to firewall engineers. No knock on other technical engineers, but the cost of getting a Checkpoint firewall engineer and the cost to maintain one QSA is a different proposition. I am not saying one is better than another technically (I’ve seen a lot of firewall engineers who could put any auditor into their place, due to their extremely proficient technical skills), I am stating the underlying cost behind the position, which is why PCI-DSS is priced at a rate that’s comparable to say, CMMI, as opposed to say, the ISO9001.

On top of just auditing cost, QSAs take into account the actual support they are giving year on year. Some of them unburden this cost to partners and consultants who have been trained (such as PKF – and there are also other matters such as independence of audit vs implementation advisory which we will discuss later), or some of them take it upon themselves. But you must know the QSAs job is not easy. Aside from auditing and supporting, there is evidence validation and report writing. Then there is the matter of undergoing the Quality Assurance process, which brings more resources/cost to the QSA company. All this while travelling to and from audit sites, reviewing etc – the life of a QSA (ask any QSA) is itinerant and often travel heavy. Burnout may also be a concern, so if the QSAs are involved in the day to day or week to week assistance to their client’s PCI program, this isn’t sustainable.

Understanding all these underlying cost will allow the procurement or whoever is evaluating to understand how to look at projects. If a QSA is pricing extremely low, the question you will need to ask is: What’s being offered? Because all QSAs have more or less the same baseline cost and if a QSA priced themselves at RM800 per man day, and they are a small shop with less than 5 QSAs, what would then be their recovery rate? 200 man days of engagement to recover their initial cost? Most procurement wouldn’t think of things like this and they would just go to their “BAFO” Best and Final Offering – but when you break it down on what is expected, then you would understand that not all PCI offerings are the same. I could simply quote a client 3 man days of QSA work for the final audit and be done. That would be the best and final offering that would win. But what about the healthchecks, the management of the evidences and how they are submitted, the quality checking, the scope optimisation process, the controls checking etc etc?

And in line with our effort estimation, one should also split the pricing into two: Audit and Consultation vs Implementation service and products.

Because if let’s say we find your Requirement 10 is completely empty, and you are thinking to purchase a QRadar SIEM to address it, you could be looking upwards of RM60,000 just to get the product in. Couple that with training for engineers, usage, hiring etc, and you are well over the six figure stage just for Requirement 10! How about testing and application reviews? If you don’t have the personnel on this, then you have to consider setting aside another RM50K etc depending on how many applications/mobile applications/ systems you have in place. So it’s highly essential to have the QSA/consultant assist you in scope reduction. Most may not view it that way, so it’s essential to find an auditor who is experienced and who looks after your interest.

Finally, understand that cost of audit/consulting would be different depending on how you go through PCI-DSS. Level 1 certification requires the effort of validating evidences, doing gap assessments and auditing and writing the RoC. Level 2 SAQ with QSA signoff is slightly easier, as there is no RoC to write while the last option of self signed SAQ without QSA is obviously a lot less costly as you are basically doing a self-signoff. Those are just broad guidelines and not how QSAs may price it, because as I say, due to variables.

You could opt to use the rule of 1/3 when it comes to estimating these costs, although your mileage may vary. For instance, if the QSA throws a RM100K audit fees (comparing it to CMMI fees) for a Level 1 Certification, then a RM60-65K (2/3 of the Level 1) for a SAQ Signoff could be reasonable; and furthermore if you just need them in for consultancy for the non QSA signoff SAQ, it could be 30K (1/3 of the level 1) or so. But note, the SAQ self signoff can be carried out entirely on your own, so the cost could be close to zero as well.

I know its a tough one to place this as pricing varies so often. We aren’t selling a product with specific hardware/software. We are selling a service that will take you through 6 months of work to cover scoping exercise, project meetings, changes, consultancy and advisory, pre-audits and post audits checks, evidence and artefacts sample validations, audit, report writing, training and all the variables in between.

Let us know if you need us to look at your PCI today, drop us a note at pcidss@pkfmalaysia.com and we will attend to you immediately!

PCI-DSS: Estimating the Effort

January 31, 2021 / / 0 Comments
pci-compliance
pci-compliance

One of the most often asked question whenever we have a first call on PCI-DSS would be: How much effort will it take? The other question would be, how much would it cost? Let’s take a look at the first question first.

Misconceptions aside, which we have written on (whether PCI is a training program, a certificate, a license, a subscription etc), effort dedicated to PCI-DSS has always been a question for clients and potential clients (and ex-clients) of ours. And I admit – it’s not an easy thing to get hold of. Because, like costing and pricing, there is so much variable. But we do have some common guideline one can take – especially when it comes to initial budgeting and estimation of effort. It’s common this is needed and although it isn’t all the time accurate, it at least provides you with some idea on how this PCI-DSS standard is approached.

Before we start – we assume that you will be undertaking PCI-DSS the proper way. We say ‘proper way’ as we’ve seen a few consultants or advisors out there that just tell our client that PCI-DSS is just sorting out documentation, and sell a bunch of policy and procedures templates for RM2K and say bye. That isn’t the proper way. That’s amounting to modern day charlatans. Or some other companies knowingly decide to do a self signoff when their controls are not in place, or have any clue what a firewall is, but still mark firewall reviews as “Compliant”. If you are planning to go down this path, good luck and God speed.

So now – the best way to look at effort is (like pricing) split it into two. This makes it less overwhelming when you are evaluating vendors, because like pricing (effort generally has a direct correlation to pricing anyway) – it can vary A LOT. We see companies that price extremely high and we see companies that price extremely low. For anyone in procurement without an iota of understanding in technical projects, it’s going to be very overwhelming. Just by looking at pricing or effort by itself on paper and not understanding what it is puts you into a position of comparing apples to oranges, or potatoes to durians. Making a decision to go for the lowest price (which is common practice in Malaysia) works if you are evaluating a product, because the specifications are standard. Not so much for PCI projects as they vary significantly. So here’s a break down for procurement who may have some challenges understanding the project. Again, this is a guideline we use to help our clients and there is no ‘standard’ approach to measuring effort for PCI-DSS.

The two main portions of PCI-DSS for effort estimation are:

a) Advisory/Consulting/Audit Services

b) Implementation/Technical Services

a) Advisory/Consulting/Audit Services

This constitute a few parts :

i) Scoping and Optimisation of Scope

This is a critical part of advisory. Scope is generally determined by the customer, but most customers have no idea what the scope is. The critical thing about scoping is that it’s easy to either miss out things to do for PCI-DSS, or to ‘over-do’ it. An example here of missing things out would be: “Oh crap, I didn’t realise we had 15 other servers in scope for PCI-DSS penetration testing, and now only 2 weeks left to the deadline.” An example of overdoing it would be: “We just purchased this wonderful DLP system for PCI-DSS for RM5 million and busted our entire technology budget for the next year and a half. Cool eh? What? PCI doesn’t mandate it? We could have other processes in place to address that control? Ooops.”

ii) Gap Assessment

Nobody starts from a zero. Well, at least that’s our experience. In some form or another, most companies already have controls in place. The purpose of this assessment is to find out how close these controls are to the baseline requirements of the standard – hence the word “gap”.

iii) Remediation Support and Pre-Audit

Not to be confused by implementation services. Remediation support is the advisory work that comes during the remediation. A lot of services will be done during the remediation period and it’s often quite overwhelming, even for someone with project management background. Evidences need to be collated and submitted in specific formats. Evidences also need to be validated first before submission, as evaluation of evidences for PCI is a key part of the whole program. Often times, this is missed out and clients just submit lock, stock and barrel whatever they have and cry foul when the whole batch is rejected and they run out of time. It’s critical for evaluation of evidences to be done properly and in a proper methodology, whether by milestone a’la Prioritized approach , or based on the QSA’s approach. A pre-audit is usually done as well to ensure clients are well and prepared for the final certification audit. This pre-audit acts like an internal audit review for ISO equivalent. A good consultant here should also provide monthly healthchecks to ensure the implementation isn’t go wayward. In fact, we spend almost 2-3 days a week with our clients onsite around a month before the actual audit starts to ensure they have everything in place for a successful audit.

iv) Certification audit and post audit

Cert audit is what the QSA does. After that, there may be a period of 2 -3 weeks to clean up whatever non-compliances found during the audit. During this time, the Report of Compliance is also prepared for level 1 clients. The process of RoC can take up to 4 – 6 weeks from the QSA, so be aware of this timeline.

b) Implementation/Technical Services

The reason this should be separated from the advisory/consultation portion is because this is actually done during stage a.iii above. It can be done by your vendor, but it also can be done internally if you have the resource. PCI-DSS doesn’t specifically require stringent standards to do services. We have customers insisting that PCI-DSS requires CREST certified penetration testers to pass. That’s simply not true at all. If you have qualified individuals (and this may not even mean they need to have certifications) who can demonstrate aptitude in doing testing in both usage of tools, experience and methodology, it’s considered acceptable, as long as they are independent enough and free from conflict of interest, for instance they shouldn’t be the application developers doing the penetration testing for the app. While it’s all fine and well to have an experienced company certified with a dozen certification to do testing, the baseline interpretation of PCI has always been agnostic to these specific certifications. So now you know. On the other hand, you also can’t just do the remediation services as and how you please. Firing up OpenSSL scanner and calling it a web application pentest won’t cut it.

There are a whole lot of other services here to be done – firewall reviews, patching, logging and monitoring, physical security, encryption, policies and procedures, web app testing, secure code review, SDLC, card data scans and the list goes on. There is a lot of work here, and how you estimate the effort should depend on what sort of gaps you get.

This is the hardest portion to estimate.

For Cert and Advisory, the effort is usually based on two factors:

a) Processes – is authorisation/settlement in scope? Is backend processes in scope? Is call center in scope? Is POS /ecommerce scoped? Is managed Service scoped? etc

b) Locations – is your DC/DR in place? Are branch offices scoped in? What about outlets (for merchants like retailers/fast food/oil and gas) etc?

The more processes in place for PCI, the more needs to be audited. The more locations, the more time. One may think, what about systems in scope? Wouldn’t auditing 10 servers vs 200 servers be vastly different? The answer is: it depends. Because technically, if you have a large amount of assets, we revert to sampling basis so we can still have a control of how many systems to audit. Some QSAs will deal with either 10-15%, but it really ranges depending on the distribution, the error variance, the type of systems, the standardisation of processes etc. So because of sampling, auditors and consultants have a measure of control over the effort required for large/small projects. Locations are similar, but locations oftentimes need a physical audit, so it’s not just remotely looking at screenshots or evidences, but actually going onsite – which requires time and effort.

For implementation/technical services, there is no sampling. A lot of confusion stems from clients thinking that implementation of controls are also on a sample basis. No. If there are 250 servers in scope, all need to have PCI controls (patched, pentested, secured, hardened). The auditor may select 20-50 systems from that set to review, but that doesn’t mean you just implement controls on 20-50 subset of the systems. So for implementation, the effort is directly related to how many assets/systems are in scope to implement. Furthermore, these should be broken down into

a) Services that can be done in-house – anything that can be done in-house, whether services or with products like logging and monitoring system etc

b) Services that require external vendors(like an ASV scan or any services you may not be able to do in-house)

c) Services that require product purchases or implementation – this is important as there would be effort for implementation, migration, testing. Somewhat similar to b), but there may be products you can actually implement yourself.

Putting it all together

Whew. That’s a lot of ground.

As you can see, the budgeting process can actually be:

Advisory/Cert Budget –> after gap assessment –> Implementation Services Budget.

Because only after the gap, would we know what we need to fix, right?

Unfortunately, procurement is often faced with the prospect of budgeting for ALL phases from the get go. This produces a lot of problems, and a LOT of variations. Procurement runs the risk (without them knowing) of getting consultants/QSA on board for advisory and cert and under-budgeting/over-budgeting for the implementation service. Any QSA/consultant worth their salt should be able to do ALL the services listed above under Advisory/Cert portion. Many QSAs only do certification only with a sprinkling of support. This is a problem because their involvement is often too late and because their price point is so low, they generally don’t do any internal advisory support, healthchecks etc. This is basically you get what you pay for concept.

As for the implementation – budgeting BEFORE knowing what is wrong is akin to giving medicine before having a diagnosis. So you could either give the right medication or the wrong one. A wrong one could be providing panadol for someone facing terminal brain cancer, or providing a liver transplant to someone having stomache from eating too much nasi lemak. Both bad.

Instead, procurement should give some standard guidelines as much as possible – number of assets estimated, number of locations, number of processes, number of firewalls, number of applications etc. The more information provided, the more accurate the effort is. Also, request for a breakdown of each, so at least you know if they quantity changes later, how much would it be more or less. Armed with this, it may be worthwhile to guestimate that the implementation cost if majority services are outsourced, would be around 1 – 1.5x the cost of the advisory. That is an extremely liberal estimate, but at least that’s what we see mostly.

We do have clients that insist on us passing them a ‘generic’ PCI cost without them providing us any information. I don’t know why. But mainly because they don’t know what’s happening. In this case, we just interpret the scope for them from an external perspective and make assumptions and send to them an estimate. But because of this, the effort and price range varies INCREDIBLY.

Remember – least effort doesn’t mean that PCI-DSS is being achieved. Because this isn’t a product, there is a huge amount of variation in effort estimation by different companies. Procurement needs to get onboard and understand the process and not just look and say – oh, why does this guy give me only 10% effort of yours? Because, Mr/Mrs Procurement, they are giving you 10% of what you need. Or, on the flip side, someone is giving you 200% more than what you need.

The next article, we will have a look at price points and see what’s really there to budget for in a PCI -DSS program. Before that, drop us a note at pcidss@pkfmalaysia.com for any enquiry and we will get back to you immediately! Be safe!

PCI-DSS: The AoC Problem

November 28, 2020 / / 0 Comments
pci-compliance
pci-compliance

Recently we were reminded once again why we constantly state that PCI-DSS must chuck away the Certification of Compliance for good. Not only it’s an unacceptable documentation to the PCI Council, but it presents a lot of problems for auditors and assessors, as well as organisations seeking PCI-DSS compliance evidence from their service providers.

Let’s go back to how PCI-DSS flows in the first place.

PCI-DSS applies to all organisations that store, process and transmit credit/debit card under the umbrella of Visa, Mastercard, Amex, JCB and Discover/Diner.

Requirement 12.8 further extends the need to manage service providers where card data is being shared, and where “they could impact the security of the customer’s cardholder data environment”. That word is key because many service providers we have spoken to retorts they are out of scope of PCI-DSS of their clients because:

a) They only provide infrastructure and has no access to card data

b) They only store physical copies of forms that are sealed in boxes and they don’t access it

c) They only provide hosting

d) They only provide customer service support

e) They only provide toilet cleaning services

Of the 5 most popular services above, only the last one, we can probably surmise, does not require PCI-DSS. The rest – not to say they are 100% applicable – would require at the very least a bit of scoping to determine if they are applicable or not for PCI. Such is the problem here.

Having established that even, say a cloud service provider that only provides IaaS, requires PCI-DSS, what is then the next problem?

We call it the problem of the AoC. Or rather, the lack-of-AoC. Or more accurately, the-refusal-of-service-providers-to-provide-AoC-since-they-already-have-the-Certificate-of-Compliance problem. Its a very long problem name, so we will just call it the Problem of the AoC.

The AoC is the Attestation of Compliance, which is basically a shortened version of the Report on compliance (ROC) or the Self Assessment Questionnaire (SAQ). So in ALL PCI-DSS Compliance, whether assessed by 3rd party or self assessed, there is an AoC. 100%.

This AoC will describe in summary what are the processes in scope of PCI-DSS AND services that are NOT in scope of PCI-DSS. This is absolute key. In Part 2 of the SAQ, it states the type of service and the name of Service included in the PCI-DSS compliance (below):

Right after that, we need to ensure there may be services being offered that for some reason is NOT assessed for PCI. An example here could be a company offering BPO services, but at the same time offering a payment gateway service. They could be PCI compliant for payment gateway but not compliant for their BPO – even though both would deal with credit cards. So we need due care in determining whether the service we are procuring from them is indeed, PCI Compliant.

This is very important. And the fact that most “Certificate of Compliance” actually does not state the scope of services under PCI-DSS, presents a problem for assessors.

We once had a very animated discussion with a large service provider providing a customer support application to our client that collected credit card information. The service provider insisted they are PCI-DSS compliant and they showed their ‘Certificate of Compliance’. The said their AoC is private and confidential and all of their customers have accepted their Certificate as proof of their compliance, which meant, we are obligated to accept it as well (according to their very animated representatives).

Now, we all know the Certificate of Compliance is as valuable as toilet paper (actually, maybe less, since toilet paper can sometimes be VERY valuable during the pandemic and panic buys) – so we insisted on them showing us their AoC. For the simple reason:

They offered the on-prem application to our client, i.e installed onsite to our client’s environment. Our client says since this application is ‘PCI-DSS’ compliant, we should not need to assess their application under Requirement 6 of PCI-DSS. Hmm.

This doesn’t sound right. The vendor kept insisting that PCI-DSS only requires them to show their Certificate, and that the information in their AoC are private and confidential and we have no right to request from them.

PCI-DSS is applicable to an environment, process and location. You can see these ALL clearly in the AoC. Not in the nonsensical and utterly useless Certificate of Compliance. Why we didn’t believe this was that, because the application was installed in our client’s environment, there shouldn’t be an instance where this application is “PCI-DSS” compliant. At most, they could claim an application to be PA-DSS compliant (or the new SSF compliant) – but that is also impossible as their application wasn’t a payment application related to settlement or authorisation – so it’s not eligible for PA-DSS! So how can this be ‘PCI-DSS Compliant’?

We were at an impasse. Because they refused to give their AoC, we refused to accept their Certificate of Compliance. They lodged a complaint, we stood firm. We were not going to pass our customer on the basis of some hocus-pocus documentation which was clearly NOT acceptable to the PCI council!

Finally, they relented, and gave us a redacted, valid AoC and telling us how wrong we were in insisting on this and we did not know what we were doing. But all we needed to see was the page above – where the scope of compliance was summarised. And in it, stated “XXXX Customer Service Cloud Solution”.

Cloud solution.

We asked the customer, did they subscribe to the cloud solution?

No, they didn’t. It was an on-prem. Installed, lock stock and barrel application into the VM managed by our client. In an environment and location secured by our client.

Wait, said the vendor. The on-prem solution is the same as the cloud solution backend they were using and have been assessed for PCI. So what was our problem? The only difference was that their ‘cloud solution’ was now installed on customer side, so this should still be acceptable.

So, well, that isn’t a cloud solution then, is it? I mean, if you have a secured safe and you put it into your high-security house, would that also mean you can put the same safe in the middle of Timbuktu somewhere and still have the same level of security? (No offense to Timbuktu, we are just using that as a reference…we should stop using it actually but oh well.) Wouldn’t the cloud solution also be assessed for its environment, processes and policies? Would this be the same on the customer end?

The point here, is that based on the AoC, we can clearly say that the PCI compliance isn’t applicable to the on-prem solution. So we still have to assess the application as it is, under Requirement 6, under the client’s PCI program.

This isn’t any ‘victory’ or whatever we can claim, but it is so extremely frustrating to waste so much time on matters that would not be any issue at all, if the problem of the AoC is resolved. Just HAVE THE AoC TO ATTEST PCI-DSS! And stop this Certificate baloney! Because of this, we end up behind schedule and we have to chase up again and again.

So, read the AoC thoroughly before you decide on a vendor/service provider – because the certificate they provide to you could very well be invalid to the services they are actually offering you. Insist on the AoC.

Drop us a note at pcidss@pkfmalaysia.com to know more about your compliance. We will respond to you immediately!

ASV Scans /= PCI Compliance

November 22, 2020 / / 0 Comments

There is an old story about a chicken and eagle. I hear this story being told by life coaches or motivational trainers trying to get through to our thick, jaded, technical skull that there is something more to life than coding and technology.

The abbreviated version is this: A farmer was walking and finds an eagle’s egg fallen out of the nest. He picks it up, brings it back to his farm, and puts it into the chicken coop. Soon, it hatches, and joins the other chickens in the farm and learns how to be a chicken, even though its an eagle. So this is where some of the version diverges.

a) The chicken and the eagle starts talking one day and the eagle notices another eagle flying high in the sky and he goes, “Dang, I wish I could be an eagle,” and his chicken-pal looks at him scornfully and says, “You are a chicken. How can you be like the king of all birds, soaring through the sky?” So the eagle keeps thinking he is a chicken and the next day he gets roasted for dinner. And the farmer finds his meat a bit tough and doesn’t taste like chicken at all. The moral here is: Don’t let your limitations inhibit you or you will end up a cooked and eaten. This is probably the original version before the other two came along below:

b) The farmer is visited by a naturalist who observes this ‘chicken’ and immediately knows he is an eagle. So he takes this chicken up to a high cliff, and throws him over, shouting: “Spread your wings and fly! Soar like the eagle you are meant to be!” And the eagle soars through the clouds and sky and become the king of all birds. The moral of the story: All of us are eagles, even if you think you are a chicken. All you need is a life coach or a motivational trainer to throw you off the ledge and you will soar. This is the preferred version for life coaches and motivational speakers. For obvious reason.

c) Same as story b) above, but instead of soaring, the naturalist throws the ‘chicken’ off the ledge, and it falls 100 feet and splatters its brains all over the bottom of the ledge and dies since it doesn’t know how to fly. And gets cooked and roasted for dinner. The moral of the story (and this is by far, our more preferred, realistic and risk-averse version): Don’t do something you may be destined for but not ready for. Or you will end up smashed, cooked and eaten.

All three versions have this theme in common: The eagle isn’t a chicken and the chicken isn’t an eagle. The chicken may have commonalities of an eagle, like wings and a beak, but just because it has those doesn’t make it an eagle.


Yes, I am aware that the anecdote above isn’t a very good illustration of the point I am trying to make, but I couldn’t think of a better one. And in a roundabout way, what I want to illustrate here is that ASV scans do not make you PCI Compliant.

We get this a lot.

A company would come and say they are PCI-compliant. Or we have a client who outsources certain portion of their operations to another company and that company comes back and shows us their ASV compliant scan and says this is all they need to show us. We (The auditors/consultants) are compelled to accept this because the ASV scans demonstrate their PCI Compliance, they say.

Let’s make a point here: ASV questions and subquestions in the SAQ D covers around 14 queries. Out of around 600. That means ASV covers 2.33% of PCI-DSS. There is a massive load of other controls and items covering PCI-DSS Other than those precious ASV quarterly scans. What about your patching? Hardening? Firewall security? HR policies? Logging and monitoring? Logical access? MFA? Hardening of systems? Anti-virus and host firewalls? What about service provider management? What about vendor default passwords? What about storage, encryption, key management? Software development? Application and penetration testing? Internal vulnerability scans? Training?

You can see how impossible it is to accept just the ASV report as an evidence of PCI compliance, much like how we cannot accept the chicken as an eagle, but yet, we are constantly berated upon that we don’t know what we are doing and that their Banks have accepted their ASV scans as a sign of PCI compliance, so we should to. But we can’t. We can’t accept 2.33% as a 100% of something. It’s simply mathematically not possible.

So there you go – banks. Why do banks perpetuate this myth that PCI compliance = ASV scans? Why? It’s 2.33% of PCI-DSS! You can’t accept something as an eagle just because it has wings and a beak! There’s really no argument about it.

Here is what 2.3% feels like:

a) The number of Jazz music of all US Music sales in 2013

b) Increase in slot machine spending in New Zealand in 2018 Q1

c) Auto parts industry against the US GDP in 2013

d) Android 6.0 Marshmallow installation for all Android devices in July 2016

e) Thats lesser than the % of freshwater we have on this planet (2.5% of water on the planet is freshwater)

I am sure there’s a lot of 2.33% out there on this planet, but the point we are making is this: It’s not compliance. It’s a small but important part of compliance but it’s not compliance. So no matter what your banks tell you, we can never accept the ASV scan as a sign of PCI compliance. It can be accepted as one of the evidences of PCI compliance amongst many, but not as an evidence of complete compliance.

Now, stop calling a chicken an eagle. Let us know about your questions for PCI or any compliance at pcidss@pkfmalaysia.com.

« Older posts Newer posts »

© 2025 PKF AvantEdge

— Up ↑