Pentesting: What I should have done

If had the luxury of talking to my past self, these are things I whished I would have done differently during the years that I performed pentesting. Some of these I eventually learned before I finished pentesting, others well, let’s just say they are much more recent. If I think of more items I’ll attempt to update the blog.

If you are a pentester and you are reading this, I hope you can benefit from them. Just make sure you evaluate if they are applicable to your situation and adjust them as required. If you are in a rush, here is the list, details can be found in the rest of this article:

• Don’t be afraid of talking to clients
• Always ask for equivalent access
• Avoid blackbox tests
• Write the report while you pentest
• Images, images & images
• Provide detection advice & POCs
• Provide reproducible POCs for your attacks (security regression tests)
• Provide scripts to fix the issue (when possible)
• Publish more
• Grasp the bigger picture
• Include what you didn’t do
• Don’t be afraid to say something was good

I’ve also included some crazy fantasies of mine, which I’ll always be wondering if they would’ve made a difference.

• Re-use reports and label them as such
• Provide the report upfront

Don’t be afraid of talking to clients
It doesn’t matter if it is about scope, budget, technology, but dear god talk to your client! You will save yourself, your client so much time! Many time I would just carry on to later find out there had been some miscommunication or the client just simply forgot to mention an important detail. The more you talk with your clients, the easier the relationship will become and the more you’ll be able to deliver a good pentest. Oh and after the 5th back and forth email you really should consider just calling them ;)

Always ask for equivalent access
Many companies have a rule that very high risk findings need to be reported immediately, that’s understandable. But why not ask for equivalent access? How are you going to further assess the impact, determine how far a real attacker could have gotten? It’s good to minimize the risks for a client, but that doesn’t excuse you from determining the impact of the vulnerability.

Avoid blackbox tests
Why on earth have I wasted so much time on things that were very obvious if all information had been provided? Yes, it feels “real” for the customer and the puzzle for myself as a pentester was pretty awesome. Yet, so much time and money was wasted. Based on my personal experience there are very few clients who actually benefit from a full blackbox test. Ask for source code, ask for directory listings, ask them to explain the environment. You’ll find more and higher risk vulnerablities, that are very beneficial to help secure your customer.

Write the report while you pentest
Or at the very least make sure you keep excellent notes! You will be missing a lot of details otherwise. Even worse, when you have to discuss the findings you won’t be able provide further details, because after a couple of days you will have forgotten part of those details!

Images, images & images
Don’t waste paragraphs, upon paragraphs on explaining complex issues. Start with an image providing the overview, concept. Then drill down with text, provide additional images to further clarify the flow. Your reader will not have the same frame of reference or knowledge that you do.

Provide detection advice & POCs
Why didn’t I provide at the very least advice and at the very best actual detection logic to the customer? This would benefit the customer as well as myself. In addition to the fact that fixing the issues a lot of times is pretty difficult for the customer, so while they are fixing the issue they could attempt to deploy some detection logic. Talk with your customer find out if such a possibility exists and provide WAF rule examples or some guidance for their SIEM. If they don’t have that, provide guidance on what logging they should enable in case they want to implement detection later down the road.

Provide reproducible POCs for your attacks
Sure, describing how to reproduce the issue is great, but why not provide a script? For a lot of issues this is possible and it enables the customer, their devs, their admins to reproduce & test the fix themselves. When your provided script doesn’t work then a re-test becomes much more useful. Too many times I’ve seen customers not really fixing the issues, which could have been identified by running a reproducible POC script instead of having me re-test it. Think of it like ‘security regression tests’

Provide scripts to fix the issue (when possible)
Why on earth didn’t I provide fix script when possible? Why did I spent countless hours describing where to click or what config file to adjust. Script it, make it reproducible in addition to explaining it. This will allow your customer to deploy it at scale, after having tested it and further tweaked the provided script. We work with computers, how come we are mostly doing the computer’s job (repetitive work) instead of doing the human job (creative work)?

Publish more
This one is tricky, since some companies have either a lot of bureaucracy to get your blog published (specially if they are bigger), conflict of interest (preventing you from publishing) or are ‘afraid’ it will benefit competitors. However, if you manage to survive those obstacles, publish as much possible without damaging your clients. It will benefit future you and the community as well. Simple things like figuring our that a special type of cookie can be used to route request and hit outdated nodes in the cluster (thus bypassing authentication) could be one of those niche things that would help out a fellow pentester. Same goes for giving back upstream patches to all kind of tools, same issues as blogging (maybe a couple of more legal challanges), but it will benefit everyone. Trust me, keeping your private version of a public tool in sync, is hell.

Ask the dumb questions!
Don’t be afraid of asking the dumb questions, like “what does that acronym mean?” or “Can you explain your setup to me, I’ve never heard of this” or “Can you provide us with the report from last year?” or “Are you sure there is no connection between those two machines, can you show me?”. You are not expected to be an omnipotent Q that knows everything. If you don’t ask you won’t learn.

Grasp the bigger picture
Not all high risk findings are created equally. Yes it could technically be a high risk findings, but talk to your client, determine if that’s actually the case for them!

Include what you didn’t do
You tested a lot of stuff and found a lot of stuff, but what didn’t you test? What should be performed in an additional test? Which parts of the website, mobile app, network remained a mysterie. Let the client know and they’ll decide if it’s worth the money to perform additional tests.

Don’t be afraid to say something was good
When you are out of luck and the target is actually pretty descent, why didn’t you include that in the report? Does it really hurt so much to state that something was actually good or secure within the boundaries of the assignment? If we never mention what clients actually do good, we’ll never re-enforce the good habits that they should further develop.

The crazy untested ones

Re-use reports and label them as such
Many times you’ll encounter that test after test you are including the same findings. Somehow the message doesn’t come across. Instead of creating new reports, re-use the old ones, label them as ‘being re-used’ and expand them with new findings. Unfortunately, I never got to test this in the wild, but I’m betting it would get the client’s attention much quicker!

Provide the report upfront
Many times, after a quick talk with the client and some looking around you can “feel” what the vulnerabilities will be. What if we provided those gut feelings upfront to a customer and gave them the option to fix them before we go in?