Okay, so in this part of the course we're going to explore the concept of audit procedures So audit procedures are activities that we can follow as auditors in order to collect, review and analyse evidence. So we're going to have a look at a few different evidence collection procedures such as carrying out interviews, document reviews, physical observations. We'll also have a good look at analysis and sampling techniques and how we can use those as well as technical verification. And then we'll also look at evidence analysis techniques such as corroboration, evaluation. and so forth. And we'll have a look at something that's become more and more relevant as time's gone by, which is the auditing of what we call virtual activities or virtual locations. And we'll see the considerations to take into account. Those situations as well. So the figure or the diagram we've got here is from ISO 19011, and this gives an overview of the audit process. And what it basically summarizes is the high-level steps in an audit. So let's just talk about some of those and how they relate to what we're about to present. So as an auditor, what are you doing? In reality, you're looking at the audit criteria, the requirements of a standard or requirements of policy or wherever it is you're auditing an organization against And you're going out and you're looking for information, sources of information that you're going to collect using different procedures, some of the procedures we're about to talk about, like sampling, like interviewing. etc. And this information that you gather we would call that audit evidence. And what we're going to do with that evidence depending on how much of it we collect and how we've collected it, is we're going to evaluate that against the criteria. In other words, against the requirements of the standard in this case. to make a judgment, a finding. In other words, a finding is do we think the requirements of the standard are met or not? There's more nuance to that, but at a high level that's what a finding is. And of course, overall, once we've got a series of findings, so let's say I've looked at ISO 2701 and I've got findings for clauses four to ten. and findings for the controls in Annex A, we'd re I'd review that with the team as a collective to come up with what we call an audit conclusion. Now usually in a certification audit that will be a conclusion about whether or not to recommend for certification. There are some other options there as well, but ultimately that's the audit process end to end. So this part of the course we're going to focus on how we gather, review and evaluate the evidence. So we'll talk about the findings and conclusions in later sections, but for now we want to focus on the evidence gathering and review techniques. Now there is something important to say which we'll be digging into a little bit further as we progress through this section, and that is the types of evidence gathering we do can depend on a couple of things whether or not there needs to be human interaction or not. So for example, some evidence gathering techniques of course will involve direct human interaction. I'm interviewing people, you know, doing a document review with the auditee, you know, carrying out sampling, for example, I'm certainly going to need to be interacting with people. Some stuff though doesn't necessarily involve direct human interaction. If I'm doing an offline document review, for example, or completing paperwork or doing stuff statistical analysis of a sample I've already collected. So of course we'd think about that. And we'd also think about the location. Do we need to perform this activity on-site, in person with people, or remotely. And it is now possible to conduct some parts or possibly all parts of an audit remotely And we'll come on to that at the end of this section and talk about some of the issues about conducting remote audits and also some of the circumstances under which we might decide to operate remotely. versus on site. But for now, whether we're on site or acting remotely, we can talk generally about some of these procedures. So what we're going to talk about here are three things. Evidence collection procedures. So these are procedures we will follow to obtain evidence. Evidence collection tools, so in other words techniques and different tools we can use such as sampling and technical verification and then what we do with our evidence when we essentially collect it like corroboration and evaluation and we'll take you through all of these with some examples about how and when you may want to consider using these in an audit context So the first place I want to start is with interviews. Throughout an audit, we'll spend a lot of time doing interviews. interviews, interviewing the audities, employees, possibly third parties, like suppliers and people like this who may be working with the organization. to gather evidence. Now obviously an interview will be a verbal exercise, but then as I'll explain shortly we'd follow that up. Now there are a few basic points I think most people would be um familiar with hopefully already about having an efficient interview. And this stands whatever interview you're doing and whoever you're interviewing and that is we want to make sure when we ask questions we avoid what we call leading questions. or influencing questions. What I mean by that is the kind of questions that lead people to an answer that they think they should give. So let me give an example Let's say I'm asking somebody about their backup procedures and I say to them, okay, let's talk about your backup procedures. Most organisations do incremental backups every day and then a full backup every week. Is that what you do here? If I say something like that, that's a lead. leading question but whether whether or not the organisation does what I've just said I've kind of led that person to believe that that might be the answer I want to hear. So I don't want to ask it that way I'd be far better off asking, okay, can you describe to me your approach to backup? And then of course we can we can do a little bit more digging depending on the answer that I hear We also need to take notes. That goes without saying, unfortunately, at least I can speak from my own memory. I'm not going to remember everything that was discussed in an interview. So we certainly want to take any notes quickly. And it's a good idea to what we call corroborate information. Corroborate information means verify it from multiple sources. So if one person's told me something in an interview, I might want to go and check that fact with another person in another interview. check that fact against documentation. And the purpose here isn't about catching people out or anything like that, but it's about validating have I understood things correctly. Because obviously one person's perspective on something may be different to another and we need to get beyond perspectives but essentially to the facts as best we can. So with that in mind and some of those principles in mind Let's just think about the types of interview that we could have in an audit. And we say there are two: general and detailed. So general interviews are usually where we would look at sort of the the more tactical level or even the more strategic level to understand how a process works. So the example given in here is about interviewing top managers management which is a good one. Another example I could give, let's say I want to understand HR processes, but before I get into the detail, for example, about how HR do a background check, for example example, I might want to have a high-level meeting with the let's say HR manager for example just to get an understanding about the general HR processes in the organisation. And then at some point I might want what's called a detailed interview. So a detailed interview is where we have an interview with a specialist in a given area to sort of dig deeper in that particular topic. So sticking with HR, if I sit down with a HR manager or administrator and I say, right, we are going to have an interview specifically focusing on the approach to background checking. That would be a detailed interview or if I said I'm going to sit down with a software developer to discuss the software development lifecycle in detail. I'm doing sort of a deep dive in a given area. So of course, usually you'll start with general interviews and then when you have the basic understanding the detailed interview would follow. Now when we think about conducting interviews, we could have interviews with individuals, one-on-one, or interviews with groups But let's stick with individual interviews for now. So the purpose of an individual interview is really you can spend time with a single person who will often be a subject matter expert hopefully in the area where we can really go looking for for detailed information on a specific topic. And one of the benefits of doing individual interviews, it's often possible to read body language Often body language can tell you whether a person is comfortable or not or um you know uh how they come across. Now obviously I can't use body language as evidence, I can't write in my report this person had bad body language but it can sort of help you you know understand uh uh the confidence if you like and whether you might want to explore elsewhere for further evidence. If there are sensitive elements of a discussion discussion you can possibly have those in an individual interview. What I mean is there may be things that people are willing to say in an individual interview that they might not feel confident to say in a group setting. So with that being said, one of the things that's important is when we have an interview with somebody, we always explain the confidentiality principle. Now I've got to be clear on how this works. The confidentiality principle doesn't mean that this is a secret interview. Any interview that takes part as part of an audit, the fact that the interview happened Who it happened with and when will be reported that is that is described in the audit notes. So it's not the fact that the interview is happening that is confidential where it is the content of that. We as auditors are not going to record an interview or quote people as saying things without agreement if that may So in other words, somebody, you know, if somebody does say something you know negative about their colleague or about the organization, you know, unless there's a reason that that is that is relevant, that's certainly not going to be repeated out. side of the interview and usually at the start of an interview we'll explain that to the uh to the interviewee. Also the beauty of individual interviews is you can adjust your questions accordingly. So you're not going to just stick to a rigid checklist you can adjust so depending on let's say the backup question depending on the answer I hear then of course I'll reframe my next question accordingly so you can be quite flexible with them. Now one of the challenges though is unfortunately with the best will in the world it's going to be difficult to individually interview all of the time. you don't necessarily have enough time. So for certain topics, it would make sense perhaps to have a group interview where you've got a number of subject matter experts or people together. Now there is a challenge as I'll get to in a moment with what we call group dynamics. But before I get to that It is a good idea with a group interview if you can do it, if there's more than one auditor, to perhaps have a couple of auditors there, one maybe doing the questioning, another for the notes. If you're able to do that. And ideally, you don't want to group more than five people because then it becomes very hard to manage. But with regards to group dynamics, there's a couple of things to talk about There is what I would call the bandwagon effect or strong leader effect. What I mean by that is let's imagine I'm interviewing a group, let's say there's four people in that group, they're all doing a similar job, and I ask a question. Let's say let's stick with the backup thing. Let's imagine these four people are administrators and I say okay can somebody take me through the approach to backup? And let's imagine one person takes the lead and they explain the answer. It could be that everybody else, because of the bandwagon effect, even if what's being said isn't entirely accurate, kind of nod, so to speak, and go along with that strong leader. And so you can sometimes get a false sense of reality through that sort of group think idea. The other potential problem which I have encountered one time is disagreements amongst the group and having to manage the dynamics of that. I remember auditing an organization at one point and they had four system administrators. This time my question was about the leavers process. So can you take me through how you ensure when somebody leaves that their access is adequately displayed? disabled and removed. One person gave me their answer and then somebody else said, you're working on an out-of-date process. That's not how we do things anymore. And somebody else chipped in with a different version of events and suddenly we had this argument going on in the room. Which one I had to manage, but two I kind of drew the conclusion that certainly there's either an inconsistent process or a lack of awareness of that process because clearly you know everybody is on a different page with this. But generally, whether you're interviewing a group or interviewing individuals, there are some uh important points to be made. And Annex A17 in ISO 19011, the guidance document for auditors, says a few things. And I'm just going to call out some of them that I think are particular Particularly important. The first one is as well as verifying that I am interviewing the right person, often when you kick off an interview, it's a good idea to say, can you tell me a little bit about your role and your background? Just because let's imagine I want to interview somebody about firewall management. I want to make sure before I ask perhaps technically related questions, I've got the right person in front of me. We also want to try and make interviewees feel at their ease. So you want to be doing the interviews, of course, during working hours in a place that's comfortable. Comfortable for them. And when I say I want the audit e to be at their ease, this is just about explaining what we're doing and what we're not doing. So I always make it clear: look, this is part of an ISMS audit. I'm looking at processes here and trying to understand processes. I always tell people I'm never commenting on anybody's performance. I'm not auditing the individual or commenting on them. I remember doing a one interview where I met with a person in HR and she kept saying to me after answering the question, she's obviously quite nervous, oh, was that the right answer? And I kept saying that there isn't a right and wrong answer. I'm just trying to get an understanding of how things work. Please don't worry. This is not about you or your role. This is just understanding about what goes on in the organisation. So you do your best to try and put people at their ease. Now of course you're going to take notes. Again that can be quite intimidating to people. So I often explain at the start, look I am going to type up some notes, I'll write things as I go along. Please be assured it's not I'm not quoting you, I'm just capturing notes for my own memory after the event. Are you okay with that? Again explaining that. that. Asking the open-ended questions we've talked about already. These are all important points. Now one thing ISO 19011 does allude to In interviewing. Interviewing in person can be very different to interviewing online. Because when you're interviewing in person in the room, things like body language and tone, etc. , it's often a lot easier to pick up than perhaps it is over a Zoom or a teams meeting and we'll talk about that a bit later on when we talk about remote um you know remote audit