An Introduction to Insider Threat Detection With Machine Learning

What Is an Insider Threat?

An insider threat is any user or individual with authorized permission to access physical or virtual corporate assets. Insider threats hold credentials that, if abused, can potentially violate the trust of the organization.

Types of Insider Threats

There are many users who can become insider threats, including temporary and permanent employees, third party business associates, external service providers, and companies providing support services.

Since each user is given different privileges, they pose different risks and are categorized accordingly:

• Careless employees — who do not follow corporate policies when using corporate assets. Careless employees are often not malicious. Rather, they are not properly educated on the consequences of their actions. They might break policies and install unauthorized software that might create new vulnerabilities or lead to exploitation.
• Inside agents — are insider threats who are approached by external threat actors. Typically, inside agents are either recruited, bribed, or threatened into stealing information and deliver it to the threat actor. Inside agents are used by threat actors as part of a larger scheme.
• Disgruntled employees — are insider threats trying to harm their current or ex-employer, typically by disrupting business operations or destroying data. Disgruntled employees are often motivated by a sense of having been wronged by their employer in some way. The disruptions and damage they cause are carried out as an act of revenge.
• Malicious insiders — are external threat actors who have gained unauthorized access to corporate assets. They leverage existing privileges to access data. Once they access the data, they often use it for their own personal gain, and either leak sensitive data or sell it on the black market.
• Feckless third-party entities — are business associates and partners that compromise the security of the organization. This typically occurs due to third-party negligence, misuse, or malicious use of corporate assets.

How Insider Threat Detection Works

Threat detection systems are created for the purpose of providing administrators with prompt discovery into various threats posing significant risks to the viability of systems and networks.

Traditionally, threat detection systems required a lot of programming, policies, and human intervention. The systems sent out many false positives which significantly damage productivity.

Today, the majority of threat detection systems are enhanced or are based on machine learning (ML) processes, which provide automation and autonomous capabilities to the system.

ML-based threat detection helps improve detection results with real-time monitoring and analysis capabilities, generates significantly fewer false positives, and sometimes automates responses.

Insider Threat Detection With Machine Learning: Seven Critical Stages

To detect insider threats, there are certain processes ML-based systems need to incorporate, including:

1. Data mining input — setting up user behavior and entity (UBE) datasets, including assets like applications, websites, file systems, networks, email systems, and metadata like user roles, access levels, work schedule, and monitoring time. Granular data can significantly improve the accuracy of the results.
2. Data classification — there are three main ways to set up data classification. You can use a predefined classification list such as PHI, PII, PFI, and code snippets. Alternatively, you can use semi-dynamic lists like file properties and origin. You can also leverage optical character recognition (OCR) technologies to discover data types on the fly. These setups can work for either supervised and unsupervised classification algorithms, which use the lists to filter raw data.
3. User profiling — providing the system with information about users, including user roles, departments, groups, and access levels. You can pull this information from employee records, human resources (HR) systems, system audit logs, Active Directory, and any other relevant sources. The system uses this information to create personalized profiles fed to behavioral models or access control and privilege management systems.
4. Behavioral models — provide different analysis results. A regression model, for example, can provide insights into future user behavior or help detect credit card fraud. A clustering algorithm, on the other hand, is used to compare compliance objectives with various business processes. There are other algorithms that have proven useful in generating effective behavioral models, such as feature extraction and density estimation.
5. Optimizing baselines — behavioral models generate a baseline, which is used to provide various types of information. A baseline can and should be optimized for specific purposes, such as assigning risk scores that fine tune the results. You can also add additional filtering layers to increase the efficiency of the algorithm and further reduce false positives.
6. Policies and rules integration — behavioral baselines help the system detect threats and trigger relevant alerts during events. You can combine this baseline with your policies and rules engine, which supports actions that proactively prevent threats. For example, an engine can warn users, notify admins, block certain actions and activities, run certain commands, and record the incident for future analysis and forensic investigation purposes.
7. Human feedback — machine learning models need to continuously learn and improve. Sometimes this requires human feedback. Typically, this process involves using system output to conduct manual threat assessments. The analyst can then provide the system with relevant information that improves the model.

Conclusion

Traditional threat detection systems generate way too many false positives to be truly effective. Machine learning models help improve threat detection processes by introducing automation and autonomous capabilities into the process.

Typically, the process involves setting up data mining and classification, as well as user profiling. Then you can create a behavioral model that continuously monitors and analyzes the operation and generates insights, prompts alerts, and performs actions that defend your network, systems, and assets.