After going through previously the nightmare of PCI-DSS Certificates, as described with considerable detail in our writeup previously, we are now faced with a new phenomenon called the Internal Audit Signoff, which is further confusing our clients.

OK, first of all, let’s do a brief recap.

How are 3 ways that PCI-DSS can be validated?

Answer :

  1. Full Report of Compliance (RoC) from QSA – Level 1 Service Providers, Level 1 Merchants
  2. Self Assessment Questionnaire (SAQ) signed off by QSA/ISA – Level 2 Merchants, (Maybe) Level 2 Service Providers
  3. Self Assessment Questionnaire (SAQ) signed off only by Merchant/Service provider – Level 3,4 Merchant, (Maybe) Level 2 Service providers

Those are the 3 endgames for PCI. And of course, the end scenario called Failure, or non-Compliance. But that isn’t cool, unless you are the type who is happy with Thanos snapping his fingers being the definite end to all things.

Now we all know item 1) requires the participation of a third party QSA/ISA to signoff on the Report of Compliance and the Attestation of Compliance. ISA here is internal security auditor. We won’t touch it this round, because this requires a whole new library of articles to discuss.

Item 2) likewise requires a third party QSA/ISA to signoff on the Self Assessment Questionnaire and the Attestation of Compliance.

Item 3) is basically, self signed – not a lot of acquirers take this seriously as basically, its anyone signing off anything they feel like. There is no validation, and sometimes, it’s akin to the CxO sticking a finger to the tongue and putting it up in the air and going, “Yeah, that feels ’bout right. Let’s sign off and say we have these controls!”

Let’s talk about item 1 and item 2.

In item 1, it’s a gimme that the QSA needs to go onsite to the locations to do an audit. I have never heard of any QSA signing off on a full RoC without actually going onsite. Maybe when our tech reaches a point where the QSA can be holographically present in a location and see what’s there without being physically there like a Jedi Force Ghost, that the PCI-SSC would accept the signoff. But by then, we could probably just tell PCI-SSC that these aren’t the companies they are looking for, and then there’s no need to do PCI.

Until then – the question is for item 2, for the QSA to signoff the SAQ, must they be onsite or they can provide a remote signoff?

Now if you ask a QSA what is the difference between 1) and 2), they would say, not much – except they don’t have to waste their time writing the tome called the Report of Compliance (ROC) for level 2. Level 2 is basically a judgement made by the QSA based on existing evidences that what is stated in the SAQ is true, or at least as much as they can have reasonable assurance on. The SAQ is not a document written by the QSA, although they can help, but in this case they are validating it. For Level 1, it’s a different story. They have to write the RoC and the work put into that reporting phase is surprisingly a lot. In comparison, it’s probably like reviewing a first term essay paper written by your senior students (SAQ Validation) versus writing the Silmarillion including the index (RoC).

However, for QSAs to conduct their audit and provide a fair opinion on the controls, they will still want to be onsite for option 2), much to the chagrin of many of my customers. Their argument here is: “Hey I am level 2, why must you come onsite??” And again, the crescendo grows that a Level 2 should have less things to worry about than Level 1 – another myth as old as us telling our children not to sleep with wet hair or else they will wake up with a storming headache.

To get to the bottom of this, we got directly from the horse’s mouth (in this case from Mastercard SDP program response: “In this scenario (describing item 2) the QSA has to be onsite. The QSA cannot simply review a RoC or SAQ without being at the location to validate that controls are actually in place.”

To be fair, the above discussion was applied to L2 Merchants (Level 2 Merchants) – those making more than 1 million volume card transactions per annum. Whether the QSA is willing to take the risk and perform an offsite review for a Level 3 or level 4, I wouldn’t know – that’s up to the QSA and the card brands I suppose. But to be absolutely safe, we would advice that all levels should be treated as such – if you need a QSA to signoff, that QSA needs to be onsite to get it done. Or use the Jedi Force Ghost. Both are acceptable to PCI-SSC I am pretty sure.

So, as an illustration, we had a request from a company, requesting us, for their location, to get the QSA to signoff remotely. Because “The Other QSA did it for us and certified us”. The other QSA meaning someone they engaged earlier.

OK – this certification term again. I am sure that did not happen – but many use the word certification for anything: actual RoC, doing the SAQ with QSA, signoff on SAQ by themselves, getting ASV scan etc…those are typical scenarios we see this certification word being thrown.

Digging further, we received a worksheet which was a typical ‘Scope’ document (you know, where they ask what sort of merchant you are, what business, how many locations, devices, whether you store card etc), and the instruction was to fill this up, send it over to the QSA and the QSA will ‘sign off’ their PCI-DSS compliance, all within 2 weeks.

QSA certified within 2 weeks, remotely, and with just the scope document, without validating any controls? No penetration testing or ASV? No Risk assessment? No review of information security policy? How?

We asked for the copy of the official signoff page (Section 3c of the AoC) but instead we got a signoff on a report from QSA stating what was scoped in and what was scoped out of PCI-DSS. A typical scope document. It’s a useful document, but it’s not a document required by the PCI SSC. In fact it doesn’t serve any purpose other than to simply state what is in scope for PCI-DSS based on the scope questionnaire (not the SAQ) provided by the QSA.

I am 100% sure the QSA meant well by this, but the problem was, there are interpretation issues. We cannot expect clients to right off the bat understand PCI-DSS and all it’s seemingly malarkey documents – the AoC, the RoC, the 9 different SAQs, the ASV scans, the partridge in the pear tree etc. So when we asked for a SAQ signed off by QSA, of course, clients will fall back to any document being signed off by QSAs. That’s why we are not big fans of the practice where clients are provided by ASV certificates just because they passed their ASV scans. They all think they are PCI certified because they have a QSA signed off document which is the ASV ‘certificate’! And the same here goes, this is simply a scope review document – almost like an internal audit report, that does not make a company PCI compliant. In fact, it is just confirming that the company MUST be PCI compliant according to the scope set.

So the moral of this story is: Not all QSA-signed off documents are valid documents for PCI-DSS. ASV scans, while valid, doesn’t make you PCI compliant. It’s only a small percentage of what you must do. Internal Audits or scope reviews like the one we saw, even signed off by the QSA, are not valid PCI-DSS documents. They do not make you PCI compliant. As PCI has explicitly stated before, the only valid PCI-SSC documentation are the AoC, SAQ, RoC and ASV scan reports (not certificates, with flowery borders and impressive cursive fonts in gold). Anything else are supplementary materials used to support the compliance, not to validate it.

For more clarity on PCI, drop us an email at pcidss@pkfmalaysia.com. We will try to sort any issues you have, and yes, we are the company you are looking for.