Hi everyone, thank you for joining me. This is section eight for ISO 27001 Lead Auditor, Evidence-Based Auditing We're going to talk about audit evidence, the different types of audit evidence, quality and reliability of audit evidence as well. So we're going to talk about the different types and how they interact with each other and their applicability. So 19011 clause 3. 9 defines audit evidence as records, statements of fact, or other information which are relevant to the audit criteria and verifiable. So audit evidence must be verifiable and can be either qualitative or quantitative. The uh the important thing about audit evidence is the verifiable part in relation to the actual evidence. So again, in a previous section we talked about Don't don't just take somebody's uh the oddity's word as fact. Uh you have to have some sort of way of validating the fact to prove that it's it's actually uh being done the way they say it is. So qualitative evidence. So that's the uh basically the concept of uh evaluating something from the standpoint of high, medium, and low. So for example, if uh if there's a risk of a breach, and we said it's a high risk of breach. then that that doesn't necessarily mean that there's going to be a breach, but there's the belief that it's a high risk as opposed to and that's qualitative As opposed to saying something like there's a 90% chance of a breach. If we said that, now we've qualified or we've quantified it with a number And generally, as an auditor or somebody who helps companies get ready for audit, I'm always a huge fan of quantitative evidence, especially when it comes to risk. So understand the difference between qualitative and quantitative. So different types of audit evidence. We have physical, mathematical, confirmative, technical. analytical, documentary, and verbal. And again, uh I hope everybody remembers when people say things, uh if it's possible to validate their their words, then you should absolutely do so. Don't don't assume that they're uh being uh a hundred percent truthful or accurate. So audit evidence does not have to be mutually exclusive either. So it's possible That you might have a uh a process, for example, with the within the audit T, and the process itself, just the mere existence of it, is audit evidence. But then the output of that process could be audit evidence for a completely different area. So it's possible to collect more than one type of evidence to uh determine the conformity or if somebody's the if the audit T is doing what they're supposed to be doing in relation to a particular uh control in the system So, for example, an external audit report is both a confirmative and documentary evidence. So we have an external audit that's confirming, but it's also documented, so it's documentary evidence The verification of the configuration of a firewall is not only physical because there's a firewall, it's also technical, but it's also analytical. And then an interview with a third party could both be confirmative and verbal. So we'll talk about physical evidence first out of that list we just had. Physical evidence is obtained through the observation or direct inspection of tangible elements. It is evidence that can be counted, inspected, examined, or observed. For example, the auditor can observe the physical access controls in the server room, or maybe the physical access controls to even get into the building. The auditor can observe the supporting utilities in the organization, and then the auditor could observe security measures across the board or maybe safety of human life measures Now, in relation to observation, it's important to understand that the auditor must observe it themselves. They can't take the uh the word that somebody else observed it and then just believe it. Observation is helpful in determining audit evidence for a particular kind of evidence because that allows for a faster response. So for example, If an auditor went to some sort of a shipping container for an organization because it's part of the inventory to make sure they're possibly counting inventory or whatever Or look inside the container to make sure that what's inside the container as listed on the packing slip is actually there. That would be an observation Other areas of observation could be walking into the building and observing how did you get inside the building? Do you have to register as a guest? Are there cameras? Are there badge entries? Are there turnstiles? There's all sorts of different things. We also have mathematical evidence. Mathematical evidence is obtained by validating the mathematical exactness of certain documents or records For example, the auditor can calculate the number of training hours relating to the ISMS provided to the personnel. They could also risk uh count the average time it takes to respond to an information security event or incident. as well as the number of nonconformities and corrective actions taken after each internal audit. Mathematical evidence is something I really like because It allows for numbers to be used. And why do we care about numbers? Because numbers can relate to percentages. Numbers can be related to dollars or revenue. So mathematical evidence is is really important from my standpoint as an auditor. Because it gives me a better understanding of the context of the organization. So for example, if we talk about training hours to the ISMS That are provided to people. I could say mathematically that you know each person went through three hours of training. There's a hundred people, so 300 hours of training was initially provided. I could even go so far as to say uh 52 phishing emails were sent out and uh 10% clicked the link or whatever, and then that 10% was uh provided with uh remedial training. So I like mathematical evidence a lot because it allows for a lot of different things. Obviously in financial audits, mathematical evidence is going to be utilized just to make sure that the numbers are right, but that makes sense. Uh how you use math as a part of uh mathematical evidence collection is really up to the auditor, but it's it's certainly uh something that could be done. Confirmative evidence, that is evidence that is obtained by the confirmation of a third party regarding one or more elements observed during the audit. Examples of confirmative evidence are the results of an intrusion test conducted by an external organization. So an external organization or intrusion test, think of that as penetration testing. Verbal statements by the external parties confirming that the organization has engaged in external communication activities. Now if you don't have verbal statements, you could have uh contractual statements that could mean uh something that along the same lines You could also have a prof professional certificate recognized by a recognized uh uh professional certification body So, for example, once you take this complete course and you pass the test and you apply for certification you'll get a professional certificate recognized by a certification body, which is PCB. So there's all sorts of different ways, but the important part in relation to confirmative evidence is it's done by a third party Not the people that you're actually auditing just by a third party. I like to use penetration tests as a really good example Because uh they're practically irrefutable. The the auditee, if you ask the auditee for examples of a penetration test or if it was even done, they're gonna hand you the same document that was handed to them So areas that can help in a auditee have an easier process for auditing is uh has to do with one of the uh other sections that we discussed, and that's dealing with the um outsourcing operations so or outsourcing services to a third party like payroll or uh onboarding and background checks and things like that So the reliability of confirmative evidence will depend on the reliability of the auditee in relation to the third party who provides the evidence. So a reputable third party is more reliable than a disreputable third party. This type of evidence is usually reliable if produced by an independent entity, so a third party, which should be external to the audite. So they're hired to come in and do a particular service or function or whatever. So, examples of mathematical evidence that could be used could be in the areas of competence. So, for example, if you had 10 people on your cybersecurity team But only three of them are certified in cybersecurity, then you could say your competence rate is 30%. Uh and the other s seven people on the team would have to get training in order to bring up that confirmative evidence And of course you wouldn't just assume they're certified, you would ask to see their certification papers. You could do segregation of networks showing that uh information services, users and groups are segregated. There's a variety of different ways you could come up with some sort of confirmative evidence. And as a part of that, you could even say, you know, a network drawing confirms that there is segmentation. There's also technical evidence characterized by the specific knowledge or expertise required to obtain it. This knowledge or expertise can be related to the organization, an activity, a process, a product, a service. or something else. Technical evidence is often, but not necessarily always, obtained with the help of a technical expert and requires conducting detailed inspections or observations of relevant operations. So again, in a previous section, we talked about competence of auditors. And I repeatedly said, if you're not fully aware of how to do something, then get a technical expert. As an auditor, you're not required to know every possible aspect of cybersecurity as a part of the audit itself, but you are expected to be able to contact people to help you out. So an example of technical evidence are number one, analysis of an IPS, so that's intrusion prevention system, analysis of IDS, intrusion detection system, and then analysis of a firewall, like maybe the access control lists. Or if there's a VPN, what VPN encryption is in use? Or how are they connecting? That could all be technical evidence So there's a lot of different ways. You can do the same, have that same concept with uh asking for configurations, uh password configuration uh for uh in uh uh Microsoft uh Active Directory or something, what's the minimum requirements? Just ask for a screenshot of that. That would be technical evidence.