I have both done pentests and received pentest reports. My observation is that the perceived severity often varies between the tester and the customer.

  • remotelove
    link
    fedilink
    arrow-up
    15
    ·
    edit-2
    4 months ago

    This is a complicated topic, actually. If you know all of this stuff, disregard. I can just share my viewpoint from being in security for over 20 years which has slowly morphed from pure engineering work to more of an engineering/business/compliance hybrid skill set.

    Context and actual risk matters. An easy exploit is bad and gives an adversary a place to pivot from in the org. However, where can the adversary pivot to after that? What resources are at risk then? (“Risk” is defined as the chance of “something” causing actual financial or reputational damage to an organization.)

    Lets say that a customer has a site hosted externally on WP Engine and the admin page is compromised. There may be company contact information loss, possibility of employee password reuse to leverage and of course, one of their public facing pages could be defaced. There is more, but just keeping it simple for now.

    Hopefully, WP Engine accounts and data is completely separate from the “meat” of the org: customer information, sensitive data, databases, etc… If that is true, the easy exploit is still easy, but the actual risk to the org is much lower and from a business perspective, the finding gets bumped down in priority.

    What I am saying is that a finding must be presented in full context. Is the finding easy to exploit but low risk or is it hard to exploit but has high risk? Is it easy to exploit and also is high risk?

    What Jr. security staff almost always forget is that “risk” is something that is determined by the business, not by the third party pentesters. Part of the job of the security and compliance teams in the org is to take a finding and connect the dots from that finding to other parts of the org. Actual risk and priority can then be assigned.

    Of all the security teams I have been part of, I can say that there are a million different ways to determine risk and a million more ways to prioritize a finding.

    What is even more difficult to process is that “severity” may just be a summary score of risk and exploit difficulty. It depends on the company and what flavor of security frameworks they use. Severity could also include time to exploit, time to detect and remediate and if and exploit attempt could even be detected.

    Good pentest reports will properly define all of its terms first. ie: What does “severity” and “risk” actually mean to the target organization? Security leadership needs to take that report and convert that into data that means something in terms of their budget. It’s a sad reality of how businesses operate, unfortunately.

    What I always see is that the business side of security is mostly ignored by jr engineers and pentesters. That isn’t bad though! Real engineering work is the meat of security and the “business side” of things is a major distraction.

    (My pet peeve is getting a pentest report with hypothetical issues where the tester couldn’t even show step 1 to prove a vulnerability is even exploitable. I now have a report with a “high severity” call out with no proof attached to it, but still have to sit in meetings with my management telling them the finding is likely bullshit.)

    • cron@feddit.orgOP
      link
      fedilink
      arrow-up
      7
      ·
      4 months ago

      Thanks for sharing your insight!

      I really like this aspect where you say that the business determines the risk, not the tester. I think this is an easy pitfall, especially for people with less experience than you ;)

      • mosiacmango@lemm.ee
        link
        fedilink
        arrow-up
        5
        ·
        edit-2
        4 months ago

        Sometimes the law determines the risk. Any critical/highs in PCI will get you speed bagged, so you sort those either way.

        Now, sometimes the sorting is “turn if off for the retest” which is just the business ignoring risk in a complicated way, but it still gets addressed in some way.