PKF AvantEdge

Category: PKF AvantEdge (Page 1 of 6)

Major Changes of PCI v4

March 6, 2024 / / 0 Comments

So now as we approach the final throes of PCI-DSS v3.2.1, the remaining 3 weeks is all that is left of this venerable standard before we say farewell once and for all.

PCI-DSS V4.0 is a relative youngster and we are already doing hours of updates with our customers on the things they need to prepare for. Don’t underestimate v4.0! While its not a time to panic, it’s also not a time to just lie back and think that v4.0 is not significant. It is.

Below is a table that provides an insight of the major changes we are facing in v4.0.

Bearing in mind that most of the requirements now start off with keeping policies updated and document roles and responsibilities, the major changes are worth a little bit of focus. In the next series of articles, we will go through each one as thoroughly as we can and try to understand the context in which it exists on.

Let’s start off the one on the top bin. Requirement 3.4.2.

Req. 3.4.2: When using remote-access technologies, technical controls prevent copy and/or relocation of PAN for all personnel, except for those with documented, explicit authorization and a legitimate, defined business need

PCI v4.0

Ok, we have underlined and emphasized a few key points in this statement. Because we feel that is important. Let’s start with what 3.4.2 applies to.

It applies to: Remote Access

It requires: Technical Controls

It must: PREVENT THE COPYING/RELOCATION

Of the subject matter: Full Primary Account Number

In v3.2.1 this was found in section 12.3.10 with slightly different wordings.

Req 12.3.10 For personnel accessing cardholder data via remote-access technologies, prohibit the copying, moving, and storage of cardholder data onto local hard drives and removable electronic media, unless explicitly authorized for a defined business need. Where there is an authorized business need, the usage policies must require the data be protected in accordance with all applicable PCI DSS Requirements.

PCI v3.2.1

I think 4.0, aside from the relocation of the requirement to the more relevant requirement 3 (as opposed to requirement 12, which we call the homeless requirement for any controls that don’t seem to fall into any other earlier requirements), reads better. Firstly, putting it in requirement 3 puts the onus on the reader to consider this as part of protection of storage of account data which is the point of Requirement 3. Furthermore, digging into the sub-requirement, 3.4 section header states: Access to displays of full PAN and ability to copy PAN is restricted.

This is the context of it, where we find the child of this 3.4 section called 3.4.2 and we need to understand it first, before we go out and start shopping for the first DLP system on the market and yell out “WE ARE COMPLIANT!”

3.4 talks about displays of FULL PAN. So we aren’t talking about truncated, or encrypted PAN here. So in theory, if you copy out a truncated PAN or encrypted PAN, you shouldn’t trigger 3.4.2. Its specific to full PAN. While we are at it, we aren’t even talking about cardholder data. A PAN is part of cardholder data, while not all cardholder data is PAN. Like the Hulk is part of the Avengers but not all Avengers are the Hulk. So if you want to copy the cardholder name or expiration date for whatever reasons like data analysis, behavioural prediction, stalking etc…this isn’t the requirement you are looking for.

Perhaps this is a good time to remind ourselves what is Account Data, Card Holder Data and Sensitive Authentication Data (SAD).

The previous v3.2.1 doesn’t actually state ‘technical controls’, which goes to say that if it’s a documentary controls, or a policy control, or something in the Acceptable Use Policy, it can also pass off as compliant. V4.0 removes that ambiguity. Of course, the policy should be there, but technical controls are specific. It has to be technical. It can’t be, oh wait, I have a nice paragraph in section 145.54(d)(i)(iii)(ab)(2.4601) in my information security acceptance document that stated this!

So these technical control(s) must PREVENT copying and relocation. Firstly just to be clear, copy is Ctrl-C and Ctrl-V somewhere else. Relocation is Ctrl-X and Ctrl-V somewhere else. Both has its problem. In copying, we will end up PAN having multiple locations of existence. In relocation, the PAN is moved, and now systems accessing the previous location will throw up an error – causing system integrity and performance issues. Suffice to say, v4.0 demands the prevention of both happening to PAN. Unless you have a need that is:

a) DOCUMENTED

b) EXPLICITLY AUTHORIZED (not Implied)

c) LEGITIMATE

d) DEFINED

When a business need is both “documented” and “defined,” it means that the requirement has been both precisely articulated (defined) and recorded in an official capacity (documented). So a list of people with access is needed for the who, why they legitimately need to access/copy/relocate PAN in terms of their business, explicitly authorized by proper authority (not themselves, obviously).

Finally, let’s talk about technical controls. Now, remember, this applies to REMOTE ACCESS. I’ve heard of clients who says, hey no worries, we have logging and monitoring in place for internal users. Or we have web application firewall in place. Or we have cloudflare in place. Or we have a thermonuclear rocket in place to release in case we get attacked. This control already implies ‘remote access’ into the environment. The users have passed the perimeter. It implies they are already trusted personnel, or contractors or service providers with properly authorized REMOTE ACCESS. Also, note that the authorization here is NOT for remote access, it is for the explicit action of copy/relocating PAN. In this case, most people would probably not have a business reason of copying/relocating PAN to their own systems unless for very specific business flow requirements. This means, only very few people in your organization should have this applied to them, under very specific circumstances. An actual real life example would be for an insurance client we have, they had to copy all transaction information, including card details in an encrypted format and put it into a removable media (like a CD-ROM) and then send it over to the Ombudsman for Financial Services as part of a regulatory requirement. That’s pretty specific.

So what passess off as a ‘technical control’? A Technical control may be as simple as to completely prevent copy/paste or cut/paste ability when accessing via remote access. This can be done in RDP or disable clipboard via SSLVPN. While I am not the most expert product specialist in remote access technologies, I can venture to say its fairly common to have these controls inbuilt into the remote access product. So, there may not be a need for DLP in that sense, as the goal here is to prevent the copying and relocation of PAN.

Now that being said, an umbrella disallow of copy and paste may not go well with some suits or C-levels who want to copy stuff to their drive to work while they are in the Bahamas. Of course. You could provide certain granular controls, depending on your VPN product or which part of the network they access. If a granular control cannot be agreed on, then a possible way is to enforce proper control via DLP (Data Loss Prevention) in endpoint protection. Or control access to CDE/PAN via a hardened jump server that has local policy locked down. So the general VPN into company resources may be more lax, but the moment access to PAN is required, 3.4.2 technical controls come in play.

At the end, how you justify your technical controls could be through a myriad of ways. The importance is of course, cost and efficiency. It has to make cost sense and it must not require your users to jump through hoops like a circus monkey.

So there you have it, a break down of 3.4.2. We are hopping into the next one in the next article so stay tuned. If you have any queries on PCI-DSS v4.0 or other related cybersecurity needs, be it SOC1 or 2, ISO27001, ISO20000, NIST or whether Apollo 11 really landed on the moon in 1969, drop us a note at avantedge@pkfmalaysia.com and we will get back to you!

Zero Trust for 2024

January 3, 2024 / / 0 Comments

As we enter into the new year, lets start off with a topic that most cybersecurity denizens would have heard of and let’s clarify it a little.

Zero Trust.

It seems a good place as any, to start 2024 off with the pessimism that accompanied the end of last year – the spate of cybersecurity attacks in 2023 had given us a taste of what is to come – insurance company – check, social security – check, the app with our vaccination information – check. While breaking down the attacks is meant for another article, what we are approaching now for the coming year is not just more of the same, but much more and more advanced attacks are bound to happen.

While Zero Trust is simply a concept – one of many – to increase resistance to attacks or breach, it’s by no means a silver bullet. There is NO silver bullet to this. We are in a constant siege of information warfare and the constant need to balance the need for sharing and the need for protection. It is as they say; the safest place would be in a cave. But that’s now living, that’s surviving. If you need to go somewhere, you need to fly, you have information with the airlines. If you need to do banking, you have information with the banks. If you need to conduct your daily shopping online, you are entrusting these guys like Lazada et al the information that otherwise you may not likely provide.

So Zero Trust isn’t the fact that you conduct zero transaction, its basically a simple principle: Trust no one, Verify everything. Compare it to the more traditional “trust but verify” approach, which assumed that everything inside an organisation’s network should be trusted, even if we do have verifications of it. Here’s a breakdown of the concept, in hopefully simpler terms.

The Basic Premise: Imagine a company as a fortified castle. In the old days, once you were inside the castle walls, it was assumed you belonged there and could roam freely. At least this is based on the limited studies we have done by binge watching Game of Thrones. All historical facts of the middle ages can be verified through Game of Thrones, including the correct anatomy of a dragon.

Back to the analogy, what if an enemy disguised as a friend managed to get inside? They would potentially have access to everything. Zero Trust Architecture operates on the assumption that threats can exist both outside and inside the walls. Therefore, it verifies everyone’s identity and privileges, no matter where they are, before granting access to the castle’s resources. The 3 keys you can remember can be:

  1. Never Trust, Always Verify: Zero Trust means no implicit trust is granted to assets or user accounts based solely on their physical or network location (i.e., local area networks versus the internet) or based on asset ownership (enterprise or personally owned). Basically, we are saying, I don’t care where you are or who you are, you are not having access to this system until I can verify who you are.
  2. Least Privilege Access: Individuals or systems are given the minimum levels of access — or permissions — needed to perform their tasks. This limits the potential damage from incidents such as breaches or employee mistakes. We see this issue a lot, whereby a C level person insist on having access to everything even if he doesn’t necessarily know how to navigate a system without a mouse. When asked why, they say, well, because I am the boss. No. In Zero Trust, in fact, because you are the boss, you shouldn’t have access into a system that does not require your meddling. Get more sales and let the tech guys do their job!
  3. Micro-Segmentation: The network is broken into smaller zones to maintain separate access for separate parts of the network. If a hacker breaches one segment, they won’t have access to the entire network.

The steps you can follow to implement the concept of Zero Trust:

Identify Sensitive Data: Know where your critical data is stored and who has access to it. You can’t protect everything. Or at least not with the budget you are given, which for most IT groups, usually is slightly more than they allocate to upkeep the company’s cat. So data identification is a must-have. Find out what is the data that you most want to protect and spend your shoe-string budget to protect it!

Verify Identity Rigorously: Use multi-factor authentication (MFA) and identity verification for anyone trying to access resources, especially important resources like logging systems, firewalls, external webservers etc. This could mean something you know (password), something you have (a smartphone or token), or something you are (biometrics). It used to cost a mortgage to implement things like this but over the years, cheaper solutions which are just as good are now available.

Contextual Access: Access decisions should consider the context. For example, accessing sensitive data from a company laptop in the office might be okay, but trying to access the same data from a personal device in a coffee shop might not be. This may not be easy, because now with mobile devices, you are basically accessing top secret information via the same device that you watch the cat playing the piano. Its a nightmare for IT security – but again, this has to have discipline. If you honestly need to access the server from Starbucks , then implement key controls like MFA, VPN, layered security and from a locked-down system.

Inspect and Log Traffic: Continuously monitor and log traffic for suspicious activity. If something unusual is detected, access can be automatically restricted. SOAR and SIEM products have advanced considerably over the years and today we have many solutions that do not require you to sell a kidney to use. This is beneficial as small companies are usually targeted for attacks, especially if these smaller companies services larger companies.

At the end, it all comes down to what are the benefits to adopt this approach.

Enhanced Security: By verifying everything, Zero Trust minimizes the chances of unauthorised access, thereby enhancing overall security. Hopefully. Of course, we may still have those authorised but have malicious intent, which would be much harder to protect from.

Data Protection: Sensitive data is better protected when access is tightly controlled and monitored. This equates to less quarter given to threat players out there.

Adaptability: Zero Trust is not tied to any one technology or platform and can adapt to the changing IT environment and emerging threats.

On the downside, there are still some challenges we need to surmount:

Complexity: Implementing Zero Trust can be complex, requiring changes in technology and culture. It’s not a single product but a security strategy that might involve various tools and technologies. This is not just a technical challenge as well, but a process and cultural change that may take time to adapt to.

User Experience: If not implemented thoughtfully, Zero Trust can lead to a cumbersome user experience with repeated authentication requests and restricted access. This is a problem we see a lot, especially in finance and insurance – user experience is key – but efficiency and security are like oil and water. Eternal enemies. Vader and Skywalker. Lex and Supes. United and Liverpool. Pineapple and Pizza.

Continuous Monitoring: Zero Trust requires continuous monitoring and adjustment of security policies and systems, which can be resource-intensive. We’ve seen implementation of SIEM and SOAR products which are basically producing so many alerts and alarms that it makes no sense anymore. These all become noise and the effects of monitoring is diluted.

In summary, an era where cyber threats are increasingly sophisticated and insiders can pose as much of a threat as external attackers, Zero Trust Architecture offers a robust framework for protecting an organisation’s critical assets. It’s about making our security proactive rather than reactive and ensuring that the right people have the right access at the right times, and under the right conditions. It’s culturally difficult, especially in Malaysia, where I will have to admit, our innate trust of people and our sense of bringing up means we always almost would open the door for the guy behind us to walk in, especially if he is dressed like the boss. We hardly would turn around and ask, “Who are you?” because we are such nice people in this country.

But, adopt we must. For any organisation looking to bolster its cybersecurity posture, Zero Trust isn’t just an option; it’s becoming a necessity. In PKF we have several services and products promoting Zero Trust – contact us at avantedge@pkfmalaysia.com and find out more. Happy New Year!

An Ode to the Invalid Certificate

April 19, 2023 / / 0 Comments

Once upon a time, in a not-so-faraway land of PeaCeEye, merchants, credit card transactions, online payments, payment gateways, POS terminals all lived in harmony. In this land, all citizens carry a trust symbol, held together by validation documents, called the Citizen Badge. However, PeaCeEye is now facing an existential threat. A threat shrouded in the cloak of validation, a false symbol of security and trust – called the Certificate. But, dear reader, beware! For this tale of caution and deception, and the Certificate, much like the elusive unicorn, while tangible, carries a false value – nothing more than a fabrication. A figment of imagination, conjured up by the minds of its idle creators, the Qessays.

You see, in the kingdom of PeaCeEye, there exists a council – a council of wise men and women who determine the rules and regulations that govern this realm. This council, known as the Secret Sorceror Council (SSC), has decreed that only three sacred documents hold the key to validation for the Citizen Badge – the Attestation of Compliance (AoC), the Report on Compliance (RoC), and the Self-Assessment Questionnaires (SAQs). Yet, despite the council’s resolute stance on this matter, a mysterious fourth document continues to emerge from the shadows – the Certificate.

Ah, the Certificate, a work of art crafted by the Qessays. You see, these Qessays were charged by the council to uphold what is truthful and right, and to ensure that all Citizens of PeaCeEye are identifiable by their Citizen Badges – The AoC, Roc and/or the SAQs. However, over the years, some of these noble Qessays have turned to the darkside and the sinister art of producing corrupted documentation, called the 4th deception, or the Certificate as it is now known. These dark Qessays have mastered the art of illusion, conjuring certificates out of thin air to dazzle their customers. They’ve become modern-day alchemists, turning mere paper and ink into a symbol of validation, which, in reality, is as weightless as a feather and as useful as a chocolate teapot. Or a fork and spoon when eating Chapati. It’s a thing of beauty, destined to hang on the walls of businesses, gracing them with its shimmering falsehoods.

But why do these Qessays continue to spin their webs of deception, offering their customers a document that has no merit in the eyes of the SSC? Something that even invalid citizens to PeaCeEye can procure? To unravel this mystery, we must dive into the murky depths of human nature. For, you see, people are drawn to shiny, pretty things, much like moths to a flame. A certificate, with its elegant calligraphy and embossed seal, is a testament to the allure of appearance over substance. It is a tangible representation of validation, regardless of its actual worth.

Moreover, the Certificate serves as a placebo, a sugar pill of sorts, which instills in businesses a false sense of security. It is a talisman that they cling to, convincing themselves that they are protected from the malicious forces of the World beyond PeaCeEye – the World called Cyberattacks. And, in the process, they become blind to the fact that the true power of validation lies in the sacred trio of documents – the AoC, RoC, and SAQs.

Now, one might argue that those who peddle these invalid certificates are merely fulfilling a demand. After all, the customer is always right, and if they desire a shiny piece of paper to adorn their walls, who are we to deny them? But, as the saying goes, “With great power comes great responsibility.” And these Qessays, as the gatekeepers of the citizenship of PeaCeEye, must hold themselves to a higher standard.

By offering these overvalued and useless certificates-that even the SSC had themselves admonished and had announced to the citizens to not place any value to them- these certificates not only betray the trust of customers but also undermine the very foundation of Citizen Badge. They turn the realm of PeaCeEye into a farce, a stage where pretenders masquerade as protectors, and businesses are lulled into a false sense of security. There are even Qessays who are not even involved in the process of validating an SAQ being answered; luring their customers to portals with questionnaires answered by the citizen themselves and then conjuring these certificates that look as if it has been validated by the Qessays, but instead are just self aggrandizing papers that has been only self validated by the person answering their own questions! In other words, the person becomes their own judge and jury and are able to produce a Certificate that looks as if they have been properly validated by a third-party Qessays. Amazing art! An ostentatious object of grandeur and magnificence, yet with all the actual value of a discarded banana peel withering in the Sahara sun.

But, dear reader, do not despair, for there is hope. You see, the truth has a funny way of revealing itself, much like the sun breaking through the clouds after a storm. And, as the truth about the invalidity of these Certificates spreads, businesses will begin to see through the veil of deception, and the demand for these counterfeit documents will wane. Qessays who persist in peddling these worthless certificates will find themselves exposed, their credibility crumbling like a house of cards.

In the meantime, we must not sit idly by, complacent in the face of falsehoods. Instead, we must raise our voices and spread the word, educating businesses on the true path to Citizen validation. We must sing the praises of the AoC, RoC, and SAQs, enlightening those who have been led astray by the allure of the invalid certificate. For it is only through knowledge that we can pierce the veil of deception and lay the mythical beast of the Certificate to rest.

So, let us embark on this crusade together, wielding the sword of truth and the shield of knowledge. As we march forward on this noble journey, let us remember the wise words of the SSC: “Trust, but verify.” Let us tear down the great wall of this Certificate, brick by brick, and replace it with a fortress built on the solid foundation of the council’s sacred trio of documents. And as we watch the last remnants of the Certificate crumble to dust, we will know that we have triumphed over the forces of deception.

We bid farewell to this Certificate, and to welcome a new era of transparency, security, and trust. An era where the mythical beast of the Certificate is relegated to the annals of history, and where the true power of validation is embraced, in all its glorious, council-approved forms. May the sacred trio of documents – the AoC, RoC, and SAQs – guide us on our path to a brighter, more secure future, and may the Certificate forever remain a cautionary tale of the perils of deception and the triumph of truth.*

** The above is written obviously in satire and tongue-in-cheek with absolute no journalistic value nor based on any real world reimagination and solely based on our absolute frustration at the continuous dependence and insistence from acquirers or banks to have our customers produce them ‘certificates’. In addition, some clients even go through self-service portals provided by QSAs and answer SAQ questions on their own, at the end of this process of self answering, a certificate is produced. Granted, the certificates do come with disclaimers in small prints stating that the certificate is actually based on self assessment and even admits that it isn’t recognised by the council.

But in reality, who actually reads the fine print?

In the end, anyone having gone through these ‘compliance’ portals, answering affirmative to everything would be able to procure these certificates and remarkably, some acquirers even accept them as proof of third party audit (which they are clearly NOT). Again, we are not stating that QSAs providing this service is doing anything wrong. There is nothing essentially wrong with certificates on its own, or QSAs providing these certificates as a simple means to show a company has undergone PCI-DSS compliance. But where it becomes a gray area is when there is too much dependence placed on these certificates to the point where even the AoC is rejected and acquirers insist on every company showing them these certificates. In this case, QSAs who are willing to provide so called certificates to companies without having undergone any assessment and only answering questions from the SAQ based on their own knowledge or whim – unless the QSA is willing to go through each question of each customer and validate these through evidence submission and review (the process called audit); then these creation of self signed certificates should be stopped. It’s akin to a banking website issuing a self-signed SSL cert on their own website and tell everyone to trust it. Does this happen in the world of e-commerce? No, it’s absurd. Then why is it different in the world of compliance? Why is this practice still allowed to prosper? How do we stop this practice?

We have been advocating removing certificates for years now from the PCI-DSS landscape and to have a more consistent and acceptable way to show PCI validation. Unfortunately, unlike the satirical tale above, this still eludes us. Drop us an email at pcidss@pkfmalaysia.com if you have any ideas and comments to this!

PCI-DSS v4.0 vs v3.2.1 Deepdive Part 1

March 17, 2023 / / 0 Comments

OK, now that we are well into 2023, the main question here is why isn’t the current assessments this year going into v4.0? Most of our customers are still doing their v3.2.1 for 2023, before doing 4.0 the next cycle. The answer is: Well, you can go for v4.0 if you want to. There’s really not much difference for now. The difference is probably more on the auditor side, as reporting requirements are different in V4.0. But from the client end, some of the scary changes like authenticated scans for internal vulnerability scanning, or updating of password complexity to 12 characters etc – these actually don’t come in force until March 2025. So there’s actually a grace period for v3.2.1 to v4.0 and another grace period for PCI v4.0 controls to be implemented, up to March 2025. Basically, anything past March 2025, the controls in v4.0 becomes Standard. No more compromise. Its like the biblical ten Commandments, except you have around 300+ commandments here. That’s a lot of chiseling on the rock by Moses.

Before we deepdive into v4.0, let’s set out the landscape a bit again, like unfurling a carpet or a mat before we feast into our metaphorical compliance picnic.

  1. Scope and Applicability

One of the key changes in PCI DSS v4.0 is the clarification of the scope of the standard. The new version provides more explicit guidance on how to apply the standard to different types of organizations, and it emphasizes the need for organizations to understand the scope of their cardholder data environment (CDE). This comes as a fairly significant change, as the initial pages of V4.0 is strewn with explanations of scoping and methodologies on how to define scope. It reads almost like they are trying to make up for lost time, and trying to cover all their bases, whereas in the previous version, just a cursory glance was done. PCI DSS v4.0 also provides guidance on how to identify and manage different types of risks. Risk has always been a difficult item to quantify in PCI. Because at the end, PCI is a result of a risk assessment anyway, done by the card schemes. It’s specifically to mitigate the risks they identified that the PCI program was born. So what’s the point of running a risk assessment in PCI-DSS if its already a standard? Well, PCI DSS v4.0 states that organizations should have a risk management program in place to identify and prioritize risks, and to take appropriate measures to mitigate those risks. Its a way of saying that while controls are required, how you address the controls are dependent on your risk assessment. Additionally, you can even opt to go above and beyond the PCI standard to address a particularly high risk area (although to find a company doing this is like finding the Lost Ark). Above the brownie points you would get from the QSA by showing you are a company keyed into your risk assessment practices; a risk assessment will likely help you identify other areas of concerns as well. The standard also requires organizations to have a process in place for identifying changes to their CDE, and for reviewing and updating their risk management program as needed. So to the point on whether the risk assessment is useful – yes. Whether it is critical to passing your PCI-DSS – well, I would say that depends a lot on your QSA. We’ve seen QSAs pass a bunch of colored coded excel sheets off as a PCI risk assessment easily.

2. New Control Objectives

PCI DSS v4.0 introduces several new control objectives to address emerging security risks. One of the key new objectives is to address the risks associated with cloud computing. The new version of the standard includes new requirements for securing cloud environments, including the need to assess the security of cloud service providers and to implement additional controls to secure cloud-based data. In v4.0, the word ‘Cloud’ appears 42 times in the entire standard. In v3.2.1, the word ‘Cloud’ appears as often as ‘NasiLemak’. Which is zero.

3. Password Requirements
PCI DSS v4.0 introduces new requirements for password management. We are in 2023 and we are still trying to remember all our passwords. PCI is now making our lives easier by introducing longer passwords! Great, now everyone just add incremental numbers behind your password from seven to twelve. The standard requires the use of multi-factor authentication for all non-console administrative access, this has already been evident in previous version. This just basically means that organizations must implement additional security measures, such as biometric authentication or smart card authentication, in addition to a password, to access sensitive systems and data

4. Encryption

The new standard maintains that organizations use more robust encryption algorithms and key lengths as per 3.2.1. Key management more or less remain as it is, but the biggest issue in v4.0 is the doing away with full disk or transparent encryption. We will do a deep dive in this later.

5. Penetration Testing and Vulnerability Management

PCI DSS v4.0 includes new requirements for penetration testing and vulnerability management. Among others is the requirement for Internal vulnerability scans to be authenticated whereas previously, this was a bit more gray area (actually not required). This could have potential impact especially for entities chasing a quarterly deadline, if you have a lot of systems in your scanning scope. So this makes the scoping a lot more critical. Because you can be sure the effort for internal scans are going to be going way up.

6. Remote Access

PCI DSS v4.0 includes new requirements for securing remote access to cardholder data environments. PCI requires organizations to implement multi-factor authentication for all remote access, and to use secure protocols, such as SSH or VPN, to access sensitive systems and data. While this remains, the other issue with 4.0 is the need to implement controls to prevent copy/relocation of PAN for all personnel unless there is a business need. We have a bad feeling about this. This could generally mean getting a DLP in place or a NAC in place to limit what can or cannot be done by users logging in remotely. There are solutions for these, but this needs to be planned and invested. The key word here is to ‘prevent’ not just ‘detect’, so this basically mean a proactive control in place to block these actions.

So in the next couple of articles, we will dive right into the changes for v4.0 in detail, including those requirements where it is stated “This requirement is a best practice until 31 March 2025, after which it will be required and must be fully considered during a PCI DSS assessment.”

We will also look into the SAQs and what has changed in the SAQs for those preparing to do self assessment in accordance to v4.0.

In the meantime, for any PCI related queries or any standards like CSA, ISO27001 etc, drop us a note at pcidss@pkfmalaysia.com and we will get back to you!

Technical Session: Clearing NTFS Dirty Bit

May 17, 2022 / / 0 Comments

Every once in a while, we take a break from boring compliance articles and write what’s more interesting – fixing broken stuff or troubleshooting problems that has nothing to do with human beings. It’s far easier dealing with machines.

So, what happened was, we had a USB plugged into one of our servers and doing some file transfers. The server wasn’t hooked up onto our UPS, as this was a test system – ok, it was actually sitting under my desk and everytime I turned it on, everyone in the office thinks a helicopter is outside the window. It’s old and loud and totally unsuitable to be located outside of a server room. Ah well.

In any case, halfway through the transfer, the power tripped. The server was ok upon restart but not the USB external drive.

It demonstrated a few symptoms:

a) When plugged in, the drive does whir up and explorer recognises it. The problem was it was listed as ‘Local Drive’ and nothing else, no other information. When clicked, it just freezes up everything. Right click does eventually brings up the context menu but when ‘Properties’ is selected, it hangs and never proceeds. So trying to scan the drive for errors from the GUI is a no go.

b) Command line wise – when accessing G:, again it just hangs. Chkdsk /f also hangs from command line so trying to scan from command line = no go.

c) Going into disk management GUI, it takes a long time before it eventually pops up and the good news was that disk management actually saw the drive. However, right clicking on it and trying to reassign the drive letter (as suggested by some other articles to recover), we get this annoying message:

The operation failed to complete because the Disk Management console view is not up-to-date.  Refresh the view by using the refresh task.  If the problem persists close the Disk Management console, then restart Disk Management or restart the computer

Microsoft being cryptic and mysterious

So like Lemmings, we proceed to refresh the console with F5 and it just hangs indefinitely and nothing happens until we unplug the drive. Then a string of errors come out like Location of drive cannot be found etc. It seems the auto opening of the USB drive was activated but Windows just couldn’t read the drive. So disk management is a no-go.

d) We tried installing other software like Acronis, or Easeus but none of these managed to read the hard drive and simply hangs until we unplug it.

e) Changing laptops/desktops/cables (all running Windows) – all the same result. The drive was acknowledged but explorer or other programs couldn’t open anything on it. This is good news actually; it doesn’t seem there was a hardware issue or any dreaded clicking noise indicating the drive was a dead duck.

f) So it does point to a software layer issue, which should be handled with a scan disk or check disk by Windows. However the problem is, the disk couldn’t be read, so it couldn’t be scanned. Booting into safe mode doesn’t help anything. Reinstalling the USB drivers doesn’t help. The drive simply refuses to go to work, like all of us on a Monday morning after being smashed with a hangover from a Sunday night out.

g) Finally, on event viewer under Windows Logs -> System, this particular classic comes up: “An error was detected on device \Device\Harddisk2\DR21 during a paging operation.” So if you go to advanced under system properties -> Performance ->Settings ->Advanced. Under virtual memory, you could uncheck the box to automatically manage the paging file size if you can. But no, Windows doesn’t read the drive, so clicking on G: once more hangs the whole system.

At this point we have wasted an hour trying to sort this nonsense out. Nothing in Windows was able to indicate the issue. One suggested running fsutil from command line. This can check for the dirty bit on NTFS, which is an annoying feature that basically renders the drive useless until the bit is ‘cleared’.

The problem with this was – yes, you got it – you couldn’t run any command on that drive as it just hangs. Nothing, no programs in Windows was able to do anything for this drive.

The Dirty Bit

So some definitions first – the dirty bit is a modified bit. It refers to a bit in memory, which switches on when an update is made to a page by computer hardware. It is just a 1 hex value situated in some place hidden on the portable hard drive.

From Microsoft definition

A volume’s dirty bit indicates that the file system may be in an inconsistent state. The dirty bit can be set because:

  • The volume is online and it has outstanding changes.
  • Changes were made to the volume and the computer was shut down before the changes were committed to the disk.
  • Corruption was detected on the volume.

If the dirty bit is set when the computer restarts, chkdsk runs to verify the file system integrity and to attempt to fix any issues with the volume. (In our case, this didn’t happen, obviously).

Assuming that this was a dirty bit problem (at this point, we were just shooting in the dark due to the lack of diagnostics, logs or events and we were just working on with some black magic of guessing).

From some articles in the net, the options to remove the dirty bit as follows:

  • You have 3 options to remove dirty bits from your computer. The first option is to trust the Microsoft disk checking utility by finishing a disk check operation. [This didn’t work as Windows wasn’t able to read ANYTHING and we could not run any windows based operations or commands or programs on it.]
  • The second method is that you move the data from the volume and format the drive. After that, move the data back. [This is way too much work. Plus, Windows can’t even access it. So the only option is to do a clone such as through Clonezilla? That’s a lot of work. And a last resort.]
  • The third method to remove the dirty bit is by using a hex editor with disk editing supported. [We didn’t explore this as this seemed a bit extreme, and probably the last time we handled a hex editor was when we had to hack in some computer games like Football Manager to give unlimited funds or a 99 in dribbling skills]

There’s an easier way.

So this is where you just need to give up on Windows and figure another way to check this disk. If you have a standby Linux box or Mac, that would help. But if not, you could actually use this great little tool called SystemRescue which among other tools, have the delectable DDRescue and Ntfs3g which will be important.

Boot up to SystemRescue (you can make a boot disk with DDRescue which is very much recommended – just use Rufus or another program to make it bootable, and download the distribution https://www.system-rescue.org/) and you basically now have a nice little Linux distro running from your USB and you should be able to also see your USB mounted with the command lsusb or lsblk.

Using lsblk -o gives you a view to see the type, size, device and a few more details. The below is an example (not ours)

Just identify which is your USB drive.

Using a the nifty ntfsfix (assuming /dev/sda1 is the USB drive you want to fix)

ntfsfix -d /dev/sda1

This basically clears the dirty bit which Windows for whatever reason, finds it so difficult to do and makes us jumps through hoops. In fact, fsutils from Windows only tells you that you have a dirty bit but doesn’t clear it. That’s like paying a doctor to tell you that you have cancer and not providing you any healthcare to it. Come on, Microsoft.

So right after clearing the dirty bit, the external drive is once again accessible. There were still some errors on the drive, but we just ran the check for errors option via GUI (since now we are able to access the properties of the drive again by right clicking for the context menu), and fixed up the inaccessible files.

So now you know. The next time you have an outage during a file transfer, it could just be the dirty bit. The problem is the diagnosis (again, Windows could just put into the event that there is a dirty bit set instead of leading us to this paging file nonsense treasure hunt). And of course, if Windows cannot access, using the SystemRescue utility, it’s a great tool to solve this issue.

And finally, according to some, another even easier way is to just plug in this drive into a Mac and apparently, it resets the dirty bit for some reason. I never tried this, so perhaps others can give it a try first before going the SystemRescue way.

Contact us at avantedge@pkfmalaysia.com for more information on what services we can offer you.

Have a good week dealing with human beings!

« Older posts

© 2024 PKF AvantEdge

— Up ↑