detection - All Articles - CISO Platform2024-03-28T19:58:38Zhttps://www.cisoplatform.com/profiles/blogs/feed/tag/detectionTop 5 Technologies To Protect Against Zero Day Malwarehttps://www.cisoplatform.com/profiles/blogs/top-5-technologies-to-protect-against-zero-day-malware2016-08-22T04:30:00.000Z2016-08-22T04:30:00.000ZAtul kumar Singh (CISO Platform)https://www.cisoplatform.com/members/AtulkumarSinghCISOPlatform<div><p>Cyber-targeted attacks such as APTs are the primary cause of concern for any organization that holds data which can be of interest to attackers. The motivations are diverse and the attackers are highly sophisticated and relentless in their approach. Traditional security tools are proving to be ineffective against such attacks as evidenced by the ubiquitous stories of successful breaches. In this time, it is considered that the more security tools you have the better secure you are which is not necessarily true.</p><p>Advance persistent threats are intelligent attacks and no matter how many controls you have in place the attacker can always learn from their failures and will eventually come up with something to evade your defenses. The key to prevent any significant damage is to strengthen your preventive controls and to have the ability to detect the attack at the earliest and respond to it swiftly.</p><p>Here in this blog we have shortlisted 5 key technologies to help you fight against Zero Day Malware.</p><p></p><p></p><p><b>Sandboxes</b></p><p>A sandbox is a security mechanism to analyze the behaviour of any suspicious file types and web objects by allowing it to execute in an isolated environment with constrained resources. It allows one to execute any untested, un-trusted/outsourced code without causing any damage to the host machine and production environment</p><p></p><p>Deployment options</p><ul><li>On-premise: Sandbox appliance is present on-premise. All the network security solutions such as firewalls, IDSes, IPSes, SWGs and SEGs feeds suspicious files into the sandbox and based on the analysis it assigns threat score for the same.</li><li>Cloud based: Sandbox appliance resides in Cloud. This deployment is very cost-effective as it reduces the cost of owning and managing appliance. Also the licensing options are flexible in this regard which further reduces the TCO.</li></ul><p></p><p></p><p><b>Big-data based Behavioral Analysis for Network traffic</b> </p><p>Network behavior analysis is particularly good for detecting new malware and zero day exploits. It is the Big-data analysis approach to solve the current security challenges related to targeted cyber attacks. The solution collects data from inside the network through sensors and other security tools and build a baseline behavior model for normal day to day chores. NBA then passively monitors the network for any anomaly in the base line behavior and if detected locates the problem point and inform the administration for further action.</p><p>NBA systems are able to detect threats against which other security tools are ineffective – for example purpose-written malware, viruses and botnets not detected by antiviruses, social engineering and other threats associated with internal network users</p><p></p><p></p><p><b>Deception technologies</b> </p><p>Deception technology is the latest armament in the fight against Advance malware and Zero-day attacks. Deception technologies deploys a network of camouflaged malware traps that are intermingled with the organizations real IT assets. the attackers will never know as the traps are identical in every way to the real IT systems. Once the attackers when compromising your network steps on one of the deception traps, a red flags is raised immediately. The Traps also analyzes the attackers tactics, techniques by keeping them occupied, giving them false information and making them feel that they are hacking into real IT assets.</p><p></p><p></p><p><b>Network forensics tools</b> </p><p>They are basically Network packet capture tools, which records and analyzes the network events in order to discover the root-cause of security incidents and other problems.According to Simson Garfinkel, there are two approaches to build a monitoring Workstation</p><ul><li>"Catch-it-as-you-can" approach: Immediately writes the packet to a disk file, Buffering in memory as necessary, and perform analysis in batches.</li><li>"Stop, look and listen" approach: Analyze the packets in memory, perform rudimentary data analysis and reduction, and write selected results to disk.</li></ul><p></p><p></p><p><b>Application virtualization</b> </p><p>Application virtualization is a technology by which any application can be made available to the end user locally without installing the application on the local computer via remote display protocol. This has many benefit other than security such as it provides central management (Patching, upgrading, migration etc), Application components are made available on demand, Reduce attack surface, mobility etc.</p><p></p><p></p></div>Incident Response Process - Signs Of Compromisehttps://www.cisoplatform.com/profiles/blogs/incident-response-process-signs-of-compromise2017-09-19T10:50:38.000Z2017-09-19T10:50:38.000Zprithahttps://www.cisoplatform.com/members/pritha<div><p>Here are some indicators which will help you detect a compromise :</p>
<ul>
<li>Identification of same email from public domain to significant number of users or C-level employees or high value targets; encrypted attachments, password protected and zipped and protected to escape email malware filter; (put user in the reference list)</li>
</ul>
<ul>
<li>End point alert / HIPS / Host based malware alerts for local script execution for the same user, raise incident</li>
</ul>
<ul>
<li>Identify usual traffic volumes to multiple ports or IP addresses or excessive packet loss (connection over 4 hours to external IP )</li>
</ul>
<ul>
<li>Examine abnormal services on known ports and abnormal ports for well-known services, verify reputation scores of IP (SSH to port 80)</li>
</ul>
<ul>
<li>EDR and WAF alerts for scripts, hash mismatch</li>
</ul>
<ul>
<li>Botnet filter alerts for traffic to blacklisted domains</li>
</ul>
<ul>
<li>Email / SPAM filter misbehavior / maintainance activity followed by suspicious activity on the network specially related to unknown / suspicious remote destinations</li>
</ul>
<ul>
<li>Monitor packet flow inside and outside from the network for likely patterns of Command and Control (C + C) traffic, outbound custom encrypted communications, covert communication channels with external entities etc.</li>
</ul>
<ul>
<li>Threat intelligence alerts for connections / data sent to suspicious destination outside organization specially belonging to less reputed geographic location and at odd hours</li>
</ul>
<ul>
<li>Examine if any data breach has occurred like large HTML packet</li>
</ul>
<ul>
<li>Review hourly and daily reports of network usage to identify unusual occurrences and spikes in traffic</li>
</ul>
<p></p>
<p></p>
<p><span>This was presented at </span><a href="https://www.sacon.io/?utm_source=CP&utm_medium=BlogPPT&utm_campaign=IRNISTKillChain_Sacon_Bang_2017" target="_blank">SACON - The Security Architecture Conference</a><span> - largest security architecture conference in the region. You can find the full presentation <a href="http://www.cisoplatform.com/profiles/blogs/incident-response-validation-containment-forensics" target="_blank">here</a>. SACON International 2017 will be hosting a Cyber Security Workshop by Dr. Phil Polstra (Author Of 'Linux Forensic').</span></p>
<p></p>
<p></p>
<p><span><a href="https://www.sacon.io/?#lp-pom-block-1401" target="_blank"><img width="600" src="{{#staticFileLink}}8669816284,original{{/staticFileLink}}" class="align-center" alt="8669816284?profile=original" /></a></span></p>
<p></p>
<p><span class="font-size-4"><strong>Dr. Phil Polstra (Author of 'Linux Forensic' & many more books) will be conducting Linux and Windows Forensic Workshop at SACON 2017. Check workshop agenda <a href="https://www.sacon.io/?#lp-pom-block-1401" target="_blank">here</a></strong></span></p>
<p></p>
<p></p></div>So, Why is Threat Detection Hard?https://www.cisoplatform.com/profiles/blogs/so-why-is-threat-detection-hard2020-10-23T17:00:00.000Z2020-10-23T17:00:00.000ZDr. Anton Chuvakinhttps://www.cisoplatform.com/members/DrAntonChuvakin<div><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">While creating a recent presentation, I needed a slide on “threat detection is hard.” And it got me thinking,<span> </span><strong class="hp il"><em class="im">why</em><span> </span>is threat detection so hard for so many organizations today?</strong><span> </span>We can trace the “cyber” threat detection to 1986 (<a href="https://en.wikipedia.org/wiki/The_Cuckoo%27s_Egg" class="co in">“Cuckoo’s Egg”</a>) and 1987 (<a href="https://www.cs.colostate.edu/~cs656/reading/ieee-se-13-2.pdf" class="co in">first IDS</a>) and perhaps even earlier events (like viruses of the early 1980s). This means we are “celebrating” ~35 years of cyber threat detection.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">However, many organizations would gladly tell you today, in 2020, that “detection is hard” for them. But why? Naturally, I posted<span> </span><a href="https://twitter.com/anton_chuvakin/status/1312107006782709761" class="co in">my draft slide on Twitter</a><span> </span>and<span> </span><a href="https://twitter.com/anton_chuvakin/status/1312105710889525250" class="co in">lively discussion ensued</a>.</p><p id="f04b" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">As I result, I updated my slide to this:</p><p><span style="font-size:24pt;">So Why Is Detection So Hard?</span></p><ul><li>Today's environments are complex & messy<br /><br /></li><li>Detection needs <strong>PEOPLE</strong><ul><li>People are hard to scale</li></ul></li><li>Detection needs <strong>DATA</strong><ul><li>Data comes from many sources, owned by many people</li><li>NEW: Context is especially key</li></ul></li><li>Detection meed <strong>TRIAGE</strong> (rules or ML)</li><li>And, eh, the attacker's font want to be detected :)<ul><li>Well, some of them ( ransomware doesn't, naturally)</li></ul></li><li><strong>NEW:</strong> Detection may be about intent, not only activity</li></ul><p></p><p>Here is a screenshot of the slide</p><div class="iv iw dj ix aj"><div class="fe ff io"><div class="jc s dj jd"><div class="je jf s"><div class="df iy t u v iz aj az ja jb"><a href="https://miro.medium.com/max/1002/0*gI-glfzZbq7ydM0x" target="_blank"><img src="https://miro.medium.com/max/1002/0*gI-glfzZbq7ydM0x?profile=RESIZE_710x" width="580" class="align-full" alt="0*gI-glfzZbq7ydM0x?profile=RESIZE_710x" /></a></div></div></div></div></div><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Now, let’s talk about it as this can be useful to those organizations that are in the beginning stages of their detection journey.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">To start, surely many people think that<span> </span><em class="im">threat detection is hard because threat actors do not want to be detected<span> </span></em>(duh!). This is an understandable, but, in my opinion, a<span> </span><em class="im">naive</em><span> </span>view. Attackers do need to remain unseen until their goals are accomplished, but the reasons for why they are unseen often have nothing to do with their craft. For sure, this argument does come up for the case of a top-tier actor facing an excellent blue/defense team. However, I’d say that other reasons below play a bigger role for most cases.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Now, my favorite top reason for why threat detection (of most/all forms) is hard: because most organization’s IT is a mess. Think sensitive data all over the place, “rogue” systems and connections, unmanaged systems and components (<a href="https://twitter.com/gepeto42/status/1312121968011825152" class="co in">good argument here</a>), layers of legacy technologies piled on top of each other (think mainframe linked to SOAP API connected to middleware and then to a mobile app). This is just bad terrain for a defender looking to spot the attacker early. BTW, perhaps belated realization of this is what gave rise to so many new asset discovery startups…</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Next, despite all the automation (SIEM, UEBA, EDR, SOAR, etc),<span> </span><strong class="hp il">many detection activities will rely on people<span> </span></strong>(and, as my<span> </span><a href="https://twitter.com/apbarros" class="co in">former favorite co-author</a><span> </span>would add,<span> process </span>too). For organization in lower tiers of the maturity scale, “people are hard,<span> </span><a href="https://blogs.gartner.com/anton-chuvakin/2012/11/08/on-buying-boxes-and-not-using-them/" class="co in">boxes are easy</a>.” People need hiring, training, retaining, morale improvement etc. Scaling teams is hard for everybody.<span> </span>Threat hunting, naturally, is<span> </span><a href="https://www.gartner.com/smarterwithgartner/how-to-hunt-for-security-threats/" class="co in">even more people-centric</a>.</p><p id="a1e5" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Next,<span> </span><strong class="hp il">detection runs on data</strong>. This does make it substantially different from “block this” or “only allow that” (and, of course, I know that some prevention runs on data too, this is not the point,<span> </span><a href="https://blogs.gartner.com/anton-chuvakin/2016/01/25/no-virginia-it-does-not-mean-that/" class="co in">this is still true</a>). Data needs to come from many sources, some incomplete and some lacking context.<span> </span><a href="https://twitter.com/ron_brash/status/1312106022580887552" class="co in">Some comments</a><span> </span>added specific points how lack of context makes detection activities hard. Very often, lacking business context does you in (<a href="https://twitter.com/shaktavist/status/1312238021647962112" class="co in">this comment</a>).</p><p id="ab7e" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Also,<span> </span><strong class="hp il">detection activities deliver signals that need to be triaged and confirmed</strong>. This partially falls into the above (detection needs people), but also touches on the inherent property of “false positives” and “false negatives.” The “false positives” need to be cleared by more technology (like IDS -> SIEM -> SOAR), people or (most likely) both. There is also overall uncertainty with finding weak signals, whether you do it with rules or with ML. </p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Sadly, teams with traditional IT mindsets often cannot work with uncertainty, inherent in our beloved domain of cyber. Hence “Need detection? Just install a detection tool!” thinking fails spectacularly.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Notice, by the way, that the data argument, the people argument and the triage argument are deeply interconnected. Detection based on incomplete or garbage data and lack of context will make triage harder and will increase the load on people too….</p><p id="cf6f" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Finally, and<span> </span><a href="https://twitter.com/AlexandreSieira/status/1312147318909489152" class="co in">this is fun, new one</a>: very often badness<span> </span><strong class="hp il">detection is about detecting intent, not the activity.<span> </span></strong>Practicality, this equates to intuition and inference yet again, something that again calls for people skills and not machines. An example: here is a connection to port 443 from this IP. Good/bad? Sure, adding context may help (What IP? What else happened? What preceded it?), but it may still prove insufficient in our attempt to deduce intent. Even “known bad” may have a good intent (ever confused a pentester for an attacker?). This does make detection even harder.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">Action items? Well, this was more of a musings post, but perhaps this:<span> </span><strong class="hp il">meditate on your threat detection mindset.<span> </span></strong>Do you crave 100% certainty? Do you expect full automation? Do you have gaps in coverage? Do you over-invest in tools over people and process? Do you think about detection as a product feature and not<span> a process</span>? These and other questions may render better results than some of the tools….</p><p id="c518" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">P.S.<span> </span><a href="https://www.cisa.gov/national-cyber-security-awareness-month" class="co in">Cyber security awareness month</a><span> </span>is here, so perhaps treat this post as my back to basics contribution…</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">P.P.S. Thanks to<span> </span><a href="https://www.linkedin.com/in/blevene" class="co in">Brandon Levene</a><span> </span>for his ever-insightful comments.</p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh"></p><p class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh">ORIGINALLY POSTED AT <a href="https://medium.com/anton-on-security" target="_blank">Anton on Security</a>.</p><p id="bd1b" class="hn ho fs hp b hq hr hs ht hu hv hw hx hy hz ia ib ic id ie if ig ih ii ij ik fl dh"></p></div>Role of Context in Threat Detectionhttps://www.cisoplatform.com/profiles/blogs/role-of-context-in-threat-detection2021-01-13T18:43:43.000Z2021-01-13T18:43:43.000ZDr. Anton Chuvakinhttps://www.cisoplatform.com/members/DrAntonChuvakin<div><p id="dfdf" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">I got into a very insightful debate with somebody who will remain nameless in the beginning of this post, but will perhaps be revealed later. The debate focused on the <strong class="hc hy">role of context in threat detection.</strong></p>
<p id="c043" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Specifically, it is about the role of <strong class="hc hy">local </strong>context (environment knowledge, organization context, site details, etc) in threat detection. <strong class="hc hy">Can threat detection work well without such local context?</strong></p>
<p id="fe13" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Now, some of you will say “yes, of course!” and will point at “<em class="hz">success</em>” (well, let’s not get into a fight over this) of anti-malware technology. After all, anti-malware tools promise to detect malware using vendor-created signatures that operate without any input from the customer about their environment (as a minor sidenote, if you “tune” AV then you do introduce that very local context). Note that for this discussion it does not matter that anti-malware will detect and then block (“prevent”) the threat (in other discussions, <a class="co ia" href="https://blogs.gartner.com/anton-chuvakin/2016/01/25/no-virginia-it-does-not-mean-that/'" target="_blank">it definitely does</a>).</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">The same line of thinking then affected intrusion detection as it was developing in the late 1990s. Intrusion detection systems (IDS) that had lots of signatures and so could detect something out of the box were “successful” (at least as a business) while those that expected customers to write signatures failed or had to evolve.</p>
<p id="a8cc" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Then it was SIEM’s turn: SIEM vendors with lots of rules and reports <a class="co ia" href="https://blogs.gartner.com/anton-chuvakin/2015/12/02/starting-a-siem-project-from-vendor-use-case-content-win-or-fail/" target="_blank">were more successful</a> (and now we have <a class="co ia" href="https://socprime.com/" target="_blank">SOC Prime</a> with lots of community rules). Next, it was <a class="co ia" href="https://blogs.gartner.com/anton-chuvakin/2017/09/06/security-analytics-platform-first-or-content-first/" target="_blank">security analytics tool sets</a> with their “trained ML unicorns”: those with lots of pre-tuned algorithms seemed to be selling better. See the pattern yet? It seems like you can be successful with threat detection without any input from each specific client.</p>
<p id="d2ec" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Now, let’s pause and think for a second! What if the industry was … well … if not wrong, but also not entirely right. <strong class="hc hy">What if truly successful threat detection must be a collaboration between the vendor and the customer?</strong></p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">In fact, it is easy to find examples of where canned and context-less threat detection does not work all that well. For this, let’s review how successful the detection technologies really are in regards to their use of local context data.</p>
<ul>
<li class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ib ic id dh"><strong class="hc hy">Anti-malware </strong>mostly works (when it does) yet the ransomware epidemic continues and top-tier state-sponsored/-affiliated malware is almost never detected by traditional anti-malware tools. Along the same line, many initial loaders (that you may call “commodity”) aren’t well detected either, and it’s easier to obtain access to these as malicious tools than ever before. Finally, when used in large enterprises, AV is often tuned hence this local knowledge is in fact introduced.</li>
<li class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh"><strong class="hc hy">Network IDS </strong>and related technologies (like NDR) don’t really work or don’t work well without local context; at the very least, you will need to “tune” (i.e. add local context like “ignore this server, it always triggers that in legitimate traffic”). Untuned NIDS has long been a subject of many jokes, dating back to the 1990s, if not the 1980s.</li>
<li class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh"><strong class="hc hy">SIEM </strong>mostly does not work without a lot of local context, vendor-written SIEM rules never became “shoot and forget”, and you need to tweak them based on your environment and/or write your own rules. This is accepted by most sane SIEM vendors and customers.</li>
<li class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh"><strong class="hc hy">EDR </strong>would be a mixed bag, in this regard. Many EDR rules are naive pattern matching. Take a powershell execution with specific command line parameters. A rule may be tuned from 22,000 results all the way down to 17 because (say) PowerShell gets executed in a “suspicious” way all the time and local context (whitelist for system, process, application, etc) is needed. With ML-based EDR, the situation is … as far as I see… the same. Anomalies detected need local context to mean something.</li>
</ul>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">(note that for attackers armed with <a class="co ia" href="https://lolbas-project.github.io/" target="_blank">“living off the land” techniques</a>, the balance skews even further towards local context criticality for detection)</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">So, what can we learn from this? <strong class="hc hy">Threat detection today needs local context a lot more than people realize. </strong>Now, successful threat detection programs at elite enterprises, especially those that follow the <a class="co ia" href="https://medium.com/anton-on-security/can-we-have-detection-as-code-96f869cfdc79" target="_blank">“detection engineering”</a> model all know this (this is why most/all of their detection logic is custom or customized, not OOB). But are they <a class="co ia" href="https://medium.com/anton-on-security/why-is-threat-detection-hard-42aa479a197f" target="_blank">a rare exception</a> rather than a trend?</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">And what does it mean for others? Well, you can hire “help” which here means an MSSP or an MDR (BTW, <a class="co ia" href="https://www.gartner.com/reviews/market/managed-detection-and-response-services" target="_blank">MDR label</a> was born out of frustration with some <a class="co ia" href="https://www.gartner.com/document/3994058" target="_blank">MSSP </a>threat detection offerings, so YMMV). However, please don’t automatically assume that “using an MSSP means that your local realities will be included in the detection process.” They will be — with quality MDRs and MSSPs, but you may also get canned off-the-shelf SIEM or even IDS alerts from some providers. You may need <a class="co ia" href="https://blogs.gartner.com/anton-chuvakin/2018/06/21/is-security-just-too-damn-hard-is-productservice-the-future/" target="_blank">a combination of tools, services</a> and — yes, still! — your own efforts.</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Finally, this is where an ML unicorn will again emerge out of the bushes (or wherever they live…) and say “but we can just auto-learn local realities using my little machine brain.” And, presumably, “auto-learn” here will not mean “import from customer repository” (because many organizations simply lack such a thing, like they lack a current and correct list of assets). Well, can it happen? Sure, it can. In theory. Personally, it is easy for me to believe that it can happen, but I will also be the first to admit that I’ve never actually seen it happen … yet.</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">So, to summarize, we all need to think ….</p>
<ul>
<li id="b7a5" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ib ic id dh">How well does threat detection really work without local context?</li>
<li id="a773" class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh">How to best include local context in various detection tools and practices?</li>
<li class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh">How to select the vendor who will detect WITH you?</li>
<li class="ha hb ff hc b hd ie hf hg hh if hj hk hl ig hn ho hp ih hr hs ht ii hv hw hx ib ic id dh">How to practice detection jointly with the vendor or service provider rather than merely “consume” it?</li>
</ul>
<p id="b64e" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">P.S. Huge thanks to <a class="co ia" href="https://www.linkedin.com/in/blevene/" target="_blank">Brandon Levene</a> for an idea for this post, for some of the examples and for a great discussion that almost became an argument :-)</p>
<p id="d5eb" class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">P.P.S. I think this situation does not really change in the cloud; you need local cloud context to detect.</p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh"> </p>
<p class="ha hb ff hc b hd he hf hg hh hi hj hk hl hm hn ho hp hq hr hs ht hu hv hw hx ey dh">Originally posted at "Anton on Security"</p></div>Top 10 SIEM Log Sources in Real Life?https://www.cisoplatform.com/profiles/blogs/top-10-siem-log-sources-in-real-life2019-08-26T22:30:00.000Z2019-08-26T22:30:00.000ZDr. Anton Chuvakinhttps://www.cisoplatform.com/members/DrAntonChuvakin<div><p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr"><a href="https://www.cisoplatform.com/profiles/blogs/top-10-siem-log-sources-in-real-life" target="_blank"><img class="align-center" src="{{#staticFileLink}}8669824063,original{{/staticFileLink}}" alt="8669824063?profile=original" width="550" /></a></p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr"> </p>
<p id="ab33" class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">One of the most common questions I received in <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2019/06/06/the-last-blog-post/" target="_blank">my analyst years</a> of covering <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/blog/category/all/?c=siem" target="_blank">SIEM</a> and other security monitoring technologies was “what data sources to <strong>integrate into my SIEM first?”</strong></p>
<p id="a2f5" class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">And of course the only honest answer to this question is: <strong class="kg ks">it depends on your security monitoring use cases </strong>and <a class="cj cf kt ku kv kw" href="https://www.gartner.com/en/documents/3844970" target="_blank"><strong class="kg ks">how you prioritize them</strong></a><strong class="kg ks">.</strong>Naturally, some people then ask <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2018/07/20/2018-popular-siem-starter-use-cases/" target="_blank">“ok, so then what are my use cases?”</a> (and then there are <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2015/11/11/fun-challenges-with-siem-use-cases/" target="_blank">these challenges</a> too). Finally, perhaps in <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2018/10/18/our-how-to-architect-and-deploy-a-siem-solution-publishes/" target="_blank">this paper</a>, we made a list of popular log sources aggregated from many organizations. Admittedly, the list may end up being useless for organizations with different security needs and challenges.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">Joking aside, big organizations often make the decision to integrate a log source into their SIEM / UEBA based on factors <strong class="kg ks">other </strong>than the pure security necessity.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">Overall, such factors may include:</p>
<ul>
<li class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr kx ky kz">Necessity for detection</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Necessity for alert triage and incident response</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Necessity as context data for another log source</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Compliance requirements to collect and retain this log type</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Compliance requirements to monitor this data source and/or system</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Ease of integration of the log source</li>
<li id="f9a1" class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Parser availability from the vendor</li>
<li class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Ability to actually transfer the log data to a SIEM</li>
<li id="a817" class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Other planned log sources that compete for attention</li>
<li id="dad5" class="ke kf bn bd kg b kh la kj lb kl lc kn ld kp le kr kx ky kz">Data volume of the log source</li>
</ul>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">And of course for users of those <em class="lf">sad SIEM products that charge per gigabyte or EPS</em> [oh… wait … this is still <em class="lf">almost </em>everybody! :-)], the <strong class="kg ks">cost of introducing a new data source into the platform</strong> may be one of the BIG deciding factors.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">Be honest: will you include a data source that will eat up 10% of your overall SIEM license if you only plan to use it as context — valuable though it may be — for another data source? Namely, if you don’t plan to write any detection rules or other logic based on this telemetry (DHCP being my favorite example here — how many detections rely solely on DHCP logs? None or very few at most).</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">As a result, my experience with SIEM deployments (going back to 2002, if you are curious) taught me that few people will include DNS or DHCP logs during their initial phases of SIEM roll-out. In fact, some will <em class="lf">never</em> include them in their SIEM! When asked why, those people explain that while they are convinced of the <em class="lf">general</em> utility of DNS logs, they do not see much value in each individual message that <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2014/06/17/on-siem-tool-and-operation-metrics/" target="_blank">costs money to collect</a>. And there are so many of those messages! Over the years, I’ve usually called them <strong class="kg ks">“sparse value logs” </strong>where the value is in getting the bulk rather than in getting some particularly valuable messages like say Windows Security Event ID 1102 …</p>
<p id="f68e" class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">As a result, SIEM operators have doubts about paying for inclusion of this data into their SIEM. The same doubt occasionally appears even for firewall logs, <a class="cj cf kt ku kv kw" href="https://blogs.gartner.com/anton-chuvakin/2016/07/21/can-i-detect-advanced-threats-with-just-flowsipfix/" target="_blank">netflow records </a>and many other high volume sources. Thus, web proxy logs, netflow, DNS, DHCP historically ended up in few SIEMs. I recall a client story from a few years back where adding web proxy logs would have 3X’d the volume of log data flowing into a SIEM. That is, web proxy logs were twice the volume of <em class="lf">all </em>other logs they collected.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">Even more so, very few people will toss all EDR telemetry into a SIEM, and usually limit themselves to EDR alerts. Admittedly, sysmon records are becoming more popular, but perhaps more so in “free” Elastic vs paid SIEM (and this will still cost you in either hardware or public cloud costs — sometimes eye-watering cloud costs at that).</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">In fact, this gave rise to an architecture where one product is used for high-value logs while another product augments it by storing more voluminous logs. However, such as architectures usually have no technical merit and bring up complexity and fragmentation and thus fragility. They do work if there are good APIs in the products (such as to query one telemetry repository from another), but it is useful to remember that they do not offer advantages other than cost.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">To summarize, in some perfect world I want to make log integration decision based ONLY on the value of such logs to my security goals and, specifically, use cases. However, today’s “popular” licensing models make this very hard.</p>
<p class="ke kf bn bd kg b kh ki kj kk kl km kn ko kp kq kr">Let’s change something!<br /><br />[cross-post from "Anton on Security"]</p></div>