Building Security Champions

Security Champions help you scale your security programs. This article describes how to build up an amazing program in your organization!

Most of us that work in cyber security are well aware that there are not enough people to fill all of the positions that we have opened. There is a severe shortage of trained and experienced people who are capable of securing the systems that we are entrusted to protect. Application security engineers, DevSecOps professionals, security architects, you name it, there's a shortage.

We will never have the staff, budget or time to do all the security work we want to do.

One of the ways that we can address this is by scaling our security teams and programs. When I say scaling, I don't mean what you do to a fish after you catch it. I mean finding a way to do more with less. This can involve automation, creating self-service systems, and many other potential solutions. In this series of blogs, we will discuss how you can solve this problem by building a security champions program for your organization.

What Is a Security Champion?

A Security Champion is a team member that takes on the responsibility of acting as the primary advocate for security within the team and acting as the first line of defense for security issues within the team.

Or, more plainly:

The person who is most excited about security on a team. They want to read the book, fix the bug, or ask security questions. Every time.

Security champions are your communicators. They deliver security messages to each dev team, teaching, sharing, and helping.

They are your point of contact, delivering messages to and from the security team and keeping you up to date on what matters to your team.

They are your advocate. They perform security work, for their dev team, with your help.

They also advocate for security, asking questions in situations you would have been left out of. Raising concerns you might have missed. They are a peer for everyone on their team and can influence in ways that you yourself cannot.

In the next few paragraphs, we will cover how to build an amazing security champions program! We will follow this recipe:

Recruit
Engage
Teach
Recognize
Reward
(Over)Communication
Metrics and Data
Conclusion (Don't Stop!)

How Does One 'Attract' Champions?

The #1 most important rule of recruiting security champions is that you must attract them. Do not "volun-tell" someone to be a security champion. That person is not going to do their best for you, and they or certainly won't enjoy the experience. Attract the right people instead of forcing them.

Performing Outreach

Use lunch and learns to teach about security.
Arrange security training.
Anyone who asks questions or attends all the events is a potential champion.
Use interesting titles for events if you can.
Add a note to your email signature saying you are looking for champions.
Put a sign on the fridge in the kitchen.
Talk about it at the all-staff meeting.
Send an email to all of IT.

Security Champions at Work!

Use lunch and learns to teach about security.
Arrange security training.
Anyone who asks questions or attends all the events is a potential champion.
Use interesting titles for events if you can.
Add a note to your email signature saying you are looking for champions.
Put a sign on the fridge in the kitchen.
Talk about it at the all-staff meeting.
Send an email to all of IT.

Observe

Pay attention to who responds, attends events, asks questions, and who is 'always there.' Those are the people you need.

Adjust Your Attitude

Change your team's mantra to "I am here to serve you," and your team will attract even more candidates. Saying "you are my customers" to the rest of IT if you are a security professional, is basically the truth. Plus, you always get more bees with honey.

#2 most important rule of recruiting: ensure their manager is on board. You don’t want this person to have to fight to do work for you or feel conflicted. Ensuring their manager is comfortable.

Engaging Your Champions

If we want IT professionals to join our security champions programs, we must make participating interesting and appealing. We want to motivate them; to do extra work on top of their regular job, to care about security, to learn a lot of new things, and to work with us. So it needs to be good.

Engage:

To occupy, attract, involve – in security activities!

To participate or become involved – with your champs!

A Few Ideas For Making Your Champions Feel Engaged

If possible, bring them on a security incident related to software. Teach them what it's like to respond, the consequences, and just how much damage insecure code can cause.

Share (appropriate) secrets with your champions. If you are going to share quite sensitive info, inform them of the concept of 'need to know, then 'Deputize' them onto your team for that one meeting. Being vulnerable and admitting mistakes is a great way to get buy-in and interest.

Let your champions see everything first. New tools, documents, policies, changes, etc. And ask their opinions. First, they will likely have great ideas, and second, it makes them feel like they matter.

Create a mailing list for your champions to tell them about new security stuff. Send them links to podcasts, articles, events, or anything else that you think is relevant and that they may find interesting.

Meet with them 1:1 once every month, and have a pre-set list of questions. Potential questions (thanks to my friend Ray at Hella Secure Blog): What are you working on? What are you going to be working on next? Do you need any help? These questions will spark conversation and lead you down the right path. That said, when you ask questions like this, brace yourself for potentially bad news so that you can play it cool if they reveal something that makes you cringe.

Hold team-building events, and let them know each other. Having a friend on a team always makes it worth coming back.

Invite them to join security communities, such as OWASP, or We Hack Purple Community (of which they are free to be part of!).

There are many ways you can make the champions feel engaged, and one of the best ones is to give them training, which is what we will talk about next.

Teaching Security Champions

You are in a room full of brand-new security champions, and they are itching to learn all about 'cyber'; what do you do? What do you teach them? How do you impress them?

Only teach them what they need to know. Nothing more.

As someone who creates security training professionally, I have to say I've seen a LOT of filler. Extra content that just does not need to be there. Software developers do not need to know the history of Diffie-Hellman or the difference between symmetric and asymmetric encryption unless they are building encryption software. So don't try to teach it to them unless they have a keen interest and have asked about it.

What they really DO need to know is:

What you need, expect and want from them, as champions.

You should define the goals of your program and share them with your champions. Share your plans for them as much as you can. Give them timelines, training information, or anything else you have. You need to clarify what you expect, or you may not get it.

Technical Topics For Teaching Your Security Champions:

Formal training on secure coding with labs!
Threat modeling
Secure architecture (whiteboarding)
Code Review
How to fix the bugs they find.
Repeat yearly as a minimum.

Topics Specific to Your Organization:

Which policies, standards, and guidelines apply to them?
Help them create missing guidelines.
Teach them how to be compliant, and help them get there.
Their role during an incident.
Job shadowing

Hold consultations to let them provide input on the policies affecting them. Trust me; their feedback will be priceless and make them feel heard.

The last topic you need to ensure they learn is tooling. If you expect them to use a tool, you need to show them how, what the output means, how to validate the results, and how to install and configure it. It is also your job to either help them pick excellent tools or involve them when you are choosing tools for them.

Recognizing and Rewarding Security Champions

Suppose you've ever read the book The 5 Love Languages or articles summarizing the five love languages. In that case, you are aware that there are predictable patterns in how people respond to various acts of kindness. Someone's "love language" is the specific type of kindness that they are most affected by. For example, someone whose love language is "words of affirmation" would respond very well to receiving a glowing performance review, a compliment on a new article of clothing, or accolades from their colleagues about a project they worked on.

You may be wondering at this point if you accidentally clicked on an article from a women's fashion magazine, not a technical article from Tanya Janca, AppSec Nerd Extraordinaire. Please have a bit of faith, and read on.

The Five Love Languages Are:

1. Gifts

2. Words of Affirmation

3. Physical Affection

4. Spending Quality Time

5. Acts of Service

When we are creating a security champions program, it's very important that we ensure the champions feel appreciated. We don't want them to feel squished into doing two jobs for only one paycheck. One of the biggest challenges that security teams face when creating a champions program is having it fall apart after the first few months, either due to the security team losing steam or champions losing interest. We need them to feel very aware of our gratitude and interest in the program itself for them to continue to want to serve the security team's agenda.

As you likely already figured out, not all the love languages listed above are work appropriate. We can't run around giving hugs or holding hands with other employees. That said, we can adapt most of them for work situations so that we can show the champions they matter to us in appropriate ways that support our security program.

Below is a non-exhaustive list of several ideas to make your champions feel as valuable as you know they are for your program.

1. (Security Related) Gifts

Physical or digital security-related gifts; books, videos, training, and CTFs.
Create a Certificate to put on their wall.
Stickers, posters, or any other decoration that is security-focused.
Tickets to a conference or training.

2. Words of Affirmation

Make sure to put a note in their performance review about them being a champion.
Tell their boss every time they do something that makes a big difference.
Send them an email and tell them when they did something big, let them know that YOU saw.
Recognize them in front of their peers (special virtual background, a star on their name is slack, etc.)
Digital badges for signature blocks.

3. Physical Affection

High Fives are the only recommended form of physical affection that you should show another employee. High fives signal success and your approval of whatever they just did.

(And only give high fives if you are confident that the employee is comfortable. If they seem hesitant, don't do it. Please also be mindful that some religions and cultures do not allow those of the opposite sex to touch each other; please be respectful if this applies. Never push physical touching at work.)

4. Spending Quality Time

Giving them your time is a reward. When you do, give them your undivided attention (put your phone away), and turn your body towards them.
Let them see a new tool first, and give them a "sneak preview" ahead of everyone else.
Let them help you make decisions. Ask for advice from them and feedback, then take it seriously.
Invite them to attend security events with you.
Whenever you meet with them, this is quality time. Ask them: What are you working on? What are you going to work on next? Do you need any help?

5. Acts of Service

Help them with more than just security. Are you good at design? Help them with it! Are you great at presentations? Offer to let them practice in front of you. You don't need to do this very often; just once can make a huge impression.
Make introductions where appropriate. "Oh yeah, Chris from QA uses that tool; I'll introduce you so you can learn."
Find answers they need to security questions and problems. Never leave them hanging.

When people feel appreciated and valued at work, they work harder (many studies show this to be true). Your champions already have full-time jobs on other teams; they are going above and beyond for you. Let them know that you are very aware of them by always making them aware of it with your actions, not just your words.

(Over)Communication With Your Security Champions

To start off with, pace yourself. Often when I speak to security teams who have a failed program, they tell me how they started off very strong. "We gave them two different pieces of training, two workshops, and three lunch and learned, all in the first three months. Then we were exhausted. We haven't done anything with them in over a year." Unfortunately, this scenario is far too common.

To pace yourself, I suggest meeting each champion monthly for 30 minutes. Then hold one lunch and learn and send one email to the champions. This might not sound like much, but you must remember they are already doing a full-time job for your organization.

In my 1:1 meetings, I like to ask the following questions (adapted from Ray Leblanc's Security Champions article on Hella Secure blog):

What are you working on?
What are you going to work on next?
Do you need any help?

Each of these questions is open-ended, with the hope that it will prompt a meaningful conversation. I usually take notes during the meeting and then send them to both of us, with any action items for either of us highlighted in bold. (Note: I've used this technique to get many of my previous bosses to do things for me. Set a reminder for a week from then, and then reply-all to that email chain and ask: "Any updates on these action items?" It works like a charm!)

In your lunch and learn (which does not need to be at lunchtime or involve food), teach them something you want them to know. Do not teach them things they do not need to know unless they asked for that topic specifically. During this session, you or a teammate can teach, or you can show them a training video you like or even a recording of a conference talk that really hit home for you. If you show them something pre-recorded, ensure you watched it first, you don't want to waste anyone's time with death-by-powerpoint. The more fun you can make these sessions, the better. If you're up for it, invite all of the developers and let everyone learn something new!

Ideas For Lunch and Learn Topics

The specifics on how to apply policies, standards, and guidelines. This could be a secure coding workshop or a threat modeling session.
Talks about the top vulnerabilities that you are seeing in your own products, including the risks they pose to your specific business model.
Workshops on how to use the tools that your team wants them to be responsible for. Especially how to configure them, how to validate results, and where to find information on how to fix what they find.
If they are responsible for design or architecture, give them secure design training.
Tell them about a security incident your team had and how it could have been prevented (assuming you are allowed to share this information).
Hold a consultation on the new policy, standard, or guideline your team is considering publishing. Ask for their feedback, then adjust your documents accordingly.
Remember to take attendance (for metrics) and take notes of any questions for you to follow up on.

The Monthly Email

Sometimes you just don't have time to do a lunch and learn event or hold 1:1s, but you still need to send a monthly email. The monthly email lets the security champions know what's going on and that they still matter to you. The program is still running because you sent an email. If you don't send this email and you haven't touched base in any other way, this leaves a space where your program may start to disappear.

The monthly email does not need to be fancy and doesn't need to say a lot. Generally, the monthly email says:

What events are happening this month at your org (lunch and learn, all staff, any other meeting they should know about)
Any updates your team has (new policy, the new tool, project updates, etc.)
Anything interesting from the news that they may find valuable
Any local security events they may be interested in
Any podcasts, videos, blog posts, or any other media that is relevant, and you feel relates to them, about security (of course)

I live in Canada, and in Canada, we are a country of immigrants. This means we have many, many different religions represented in most workplaces. In December, there's Hannukah, Ramadan, Christmas, and more, and often people take time off for these special holidays. This means having a large meeting in December is darn near impossible. This is the type of situation where you just send the monthly email! It could say something like the following:

Hello Security Champions!

As it is December and many of you will be off celebrating various holidays, we are not going to have any events this month. We also want to wish you happy holidays, and we hope you enjoy all the snow we got this past weekend!

In January we are going to boot the Champions program back up with a lunch and learn on XSS. As some of you are aware, we’ve found it in about 1/3 of our custom apps, and we want to stomp it out in the new year (with your help of course!) An invitation will arrive later this week.

In the meantime, please check out this XSS Deep Dive by Tanya Janca. We’re going to cover this topic a bit differently than she does, but it gives you a good idea of what we are up against.

Have a great December folks!

Sincerely,

The Security Team

My hope for this section of the article is that you remember to continue communicating with your champions. Don't let your program slip; it will disappear faster than you think. When in doubt, send them an email and check-in.

Metrics and Data

You may wonder why metrics are important. The answer is twofold.

1. We can use data and metrics to report to our bosses and show them we are succeeding. It's evidence of what we are doing and how well it works. You can then use that data again to ask for more resources (staff, tools, budget), a raise, or other changes.

2. The second reason is so that we, ourselves, can improve. We want to improve our program, ourselves, and our results. We can see which activities or methods produce better results when we measure our activities and their impacts. We can then use that information to change our approach for the better.

It is important, however, that we do not become fooled by vanity metrics. Vanity metrics are numbers that make us look good but don't necessarily mean anything. My talk on this subject has several stories, but let's tell one.

I used to work somewhere, and we all wrote blog posts. We were measured on how many “clicks” we got. A colleague of mine got 10X the number of clicks that I did, and I asked him how he did it. He explained he got the most clicks on Reddit. I was unfamiliar with the platform but thought I would give it a try. First though, I asked for extra data: I wanted to know how long people were staying on our articles. It turned out that people were staying on my articles approximately 1.5 minutes (which means they were reading the whole thing), and on his they were staying an average of 1.5 seconds (which means almost no one was reading the article, they were just clicking the link. This is commonly known as a “bounce”.) The purpose of our jobs was to write articles to help customers know how to use our products, and this means a bounce wasn’t valuable. Armed with this new information, we started comparing different platforms, and it turned out almost all traffic from Reddit were ‘bounces’. I also noticed that my Twitter followers were significantly more likely to read the article when compared to LinkedIn, and LinkedIn got better results than Reddit. My colleague started focussing on sharing links on Twitter (he had more followers than I did), and I started trying to get more followers on the same platform. It turns out that measuring clicks was a vanity metric. The rest, as they say, is history.

Now for your security champion program metrics! Measure the following things to see what's working and what's not. Don't forget to report upwards about the ROI (return on investment) your champions program has produced!

How many new security champions have you attracted?
Measuring program engagement: how many people attended an event, how many people reported issues to you, how many people asked questions.
Use the bug tracker for metrics on how many security bugs are being reported and fixed, especially if you have targeted a specific bug class. Also, count how many new instances of that type of bug appear; hopefully, this number will be very low.
Instances where champions have told you about a security issue you would not have known about otherwise.
If the champions report better work satisfaction and/or fewer missed days of work.
Gather stories of your champs saving the day, providing help to their teammates, or anything else that makes for a good story-telling session for upper management.

Conclusion: Security Champions

Here Are a Few More Tips to Conclude This Long Article:

First, start by defining the focus of your program and what is expected from champions. Be realistic; you can only expect one to four hours of maximum effort from them per week.
If someone takes a security course but is not on the security team, they may make a good champion. Reach out and introduce yourself.
If the mantra of the security team is "it's my job to help you do your job securely," "you're my customer," or "I'm here to serve you," that is very attractive; however, if your team is known as 'the ministry of NO!', you will have difficulty attracting volunteers until you turn over a new leaf.
Record every group session and save them. Then, create an onboarding set of champion videos from these recordings so that you can auto-onboard new champions. Some of the videos can also be used to onboard new software developers or other IT staff.
Save all the videos so anyone who missed them can see them later. Then, offer up the list of videos to everyone at your organization, if appropriate.
Include a TTT (train the trainer) package so your security champions can train their teams as needed. For instance, if you want your champions to give training or talk to their teams, have them follow your package. The package should contain 1) your slides, 2) demo information and instructions to set it up, 3) a video of you giving the talk/training, and 4) a video of you explaining what you are trying to get across for each slide and the entire demo, speaking as though you are teaching someone to give the talk on your behalf. For an example of this, see mine!
PS... Feel free to give these talks yourself at your workplace.

Lastly, don't stop. Don't give up. Perseverance is the thing that will make this program work. As your program continues, it will grow, and the value that you receive from it will also grow, scaling upwards over time. You and your organization can do this; all it takes is dedication and time.

We Provide consulting, implementation, and management services on DevOps, DevSecOps, Cloud, Automated Ops, Microservices, Infrastructure, and Security

Services offered by us: https://www.zippyops.com/services

Our Products: https://www.zippyops.com/products

Our Solutions: https://www.zippyops.com/solutions

For Demo, videos check out YouTube Playlist: https://www.youtube.com/watch?v=4FYvPooN_Tg&list=PLCJ3JpanNyCfXlHahZhYgJH9-rV6ouPro

If this seems interesting, please email us at [email protected] for a call.